-- Logs begin at Wed 2022-08-10 22:00:58 UTC, end at Wed 2022-08-10 23:16:07 UTC. -- Aug 10 22:33:26.725008 dv-cinder-836083-12-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 10 22:33:26.733176 dv-cinder-836083-12-iscsi devstack@keystone.service[93426]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 10 22:33:26.735761 dv-cinder-836083-12-iscsi devstack@keystone.service[93426]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 10 22:33:26 2022] *** Aug 10 22:33:26.735893 dv-cinder-836083-12-iscsi devstack@keystone.service[93426]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 10 22:33:26.735998 dv-cinder-836083-12-iscsi devstack@keystone.service[93426]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 10 22:33:26.736102 dv-cinder-836083-12-iscsi devstack@keystone.service[93426]: nodename: dv-cinder-836083-12-iscsi Aug 10 22:33:26.736204 dv-cinder-836083-12-iscsi devstack@keystone.service[93426]: machine: x86_64 Aug 10 22:33:26.736838 dv-cinder-836083-12-iscsi devstack@keystone.service[93426]: clock source: unix Aug 10 22:33:26.736943 dv-cinder-836083-12-iscsi devstack@keystone.service[93426]: pcre jit disabled Aug 10 22:33:26.737043 dv-cinder-836083-12-iscsi devstack@keystone.service[93426]: detected number of CPU cores: 4 Aug 10 22:33:26.737142 dv-cinder-836083-12-iscsi devstack@keystone.service[93426]: current working directory: / Aug 10 22:33:26.737242 dv-cinder-836083-12-iscsi devstack@keystone.service[93426]: detected binary path: /usr/bin/uwsgi-core Aug 10 22:33:26.737359 dv-cinder-836083-12-iscsi devstack@keystone.service[93426]: your processes number limit is 47680 Aug 10 22:33:26.737454 dv-cinder-836083-12-iscsi devstack@keystone.service[93426]: your memory page size is 4096 bytes Aug 10 22:33:26.737567 dv-cinder-836083-12-iscsi devstack@keystone.service[93426]: detected max file descriptor number: 2048 Aug 10 22:33:26.737665 dv-cinder-836083-12-iscsi devstack@keystone.service[93426]: lock engine: pthread robust mutexes Aug 10 22:33:26.737913 dv-cinder-836083-12-iscsi devstack@keystone.service[93426]: thunder lock: enabled Aug 10 22:33:26.738224 dv-cinder-836083-12-iscsi devstack@keystone.service[93426]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 10 22:33:26.738396 dv-cinder-836083-12-iscsi devstack@keystone.service[93426]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 10 22:33:26.777550 dv-cinder-836083-12-iscsi devstack@keystone.service[93426]: Python main interpreter initialized at 0x5653b13b99c0 Aug 10 22:33:26.777550 dv-cinder-836083-12-iscsi devstack@keystone.service[93426]: python threads support enabled Aug 10 22:33:26.777550 dv-cinder-836083-12-iscsi devstack@keystone.service[93426]: your server socket listen backlog is limited to 100 connections Aug 10 22:33:26.777550 dv-cinder-836083-12-iscsi devstack@keystone.service[93426]: your mercy for graceful operations on workers is 90 seconds Aug 10 22:33:26.777908 dv-cinder-836083-12-iscsi devstack@keystone.service[93426]: mapped 403077 bytes (393 KB) for 2 cores Aug 10 22:33:26.777983 dv-cinder-836083-12-iscsi devstack@keystone.service[93426]: *** Operational MODE: preforking *** Aug 10 22:33:26.778239 dv-cinder-836083-12-iscsi devstack@keystone.service[93426]: *** uWSGI is running in multiple interpreter mode *** Aug 10 22:33:26.778239 dv-cinder-836083-12-iscsi devstack@keystone.service[93426]: spawned uWSGI master process (pid: 93426) Aug 10 22:33:26.778440 dv-cinder-836083-12-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 10 22:33:26.779328 dv-cinder-836083-12-iscsi devstack@keystone.service[93426]: spawned uWSGI worker 1 (pid: 93438, cores: 1) Aug 10 22:33:26.779328 dv-cinder-836083-12-iscsi devstack@keystone.service[93426]: spawned uWSGI worker 2 (pid: 93439, cores: 1) Aug 10 22:33:26.779482 dv-cinder-836083-12-iscsi devstack@keystone.service[93426]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 10 22:33:32.373048 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 22:33:32.373048 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: from cryptography.utils import int_from_bytes Aug 10 22:33:32.373871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 22:33:32.373871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: from cryptography.utils import int_from_bytes Aug 10 22:33:32.383842 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 22:33:32.383842 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: from cryptography.utils import int_from_bytes Aug 10 22:33:32.384311 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 22:33:32.384311 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: from cryptography.utils import int_from_bytes Aug 10 22:33:32.423661 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 10 22:33:32.423864 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 10 22:33:32.424062 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] command line args: [] {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 10 22:33:32.424347 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 10 22:33:32.424555 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ================================================================================ {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 10 22:33:32.424864 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] admin_token = **** {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.425179 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] config_dir = [] {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.428182 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.428182 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] config_source = [] {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.428182 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] debug = True {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.428182 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.428182 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.428182 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] insecure_debug = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.428182 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.428829 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.428829 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] list_limit = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.428829 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] log_config_append = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.428829 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.428829 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] log_dir = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.429091 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] log_file = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.429354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.429580 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.429890 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.430209 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.430418 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.430614 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.430808 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.431572 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.431572 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.431720 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.431920 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.432214 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.432521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.432737 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 10 22:33:32.432830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] notification_format = cadf {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.432952 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 10 22:33:32.433181 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] command line args: [] {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 10 22:33:32.433258 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.433573 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 10 22:33:32.433775 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ================================================================================ {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 10 22:33:32.433865 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.76/identity {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.434002 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] publish_errors = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.434101 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] admin_token = **** {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.434299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.434448 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] config_dir = [] {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.434592 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.434670 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.434803 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.434887 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] config_source = [] {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.435084 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] debug = True {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.435181 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.435318 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.435431 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.435623 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.435706 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] standard_threads = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.435840 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] insecure_debug = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.435950 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] strict_password_check = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.436209 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.436293 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.436451 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] use_eventlog = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.436558 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.436649 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] use_journal = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.436856 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] use_json = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.436939 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] list_limit = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.437064 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] use_stderr = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.437146 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] log_config_append = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.437272 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] use_syslog = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.437409 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.437488 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] log_dir = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.437583 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] watch_log_file = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.437700 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] log_file = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.437904 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.438016 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.438206 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.438370 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.438570 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.438723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.438889 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.439101 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.439240 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.439324 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.439472 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.439556 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.439811 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.439911 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.440137 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.440343 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.440431 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.440707 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.440853 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.440962 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.441045 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.441217 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.441432 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.441514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.441752 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.441932 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.442071 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.442234 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.442312 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] notification_format = cadf {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.442485 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.442673 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.442758 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.442969 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.443230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.443477 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.76/identity {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.443567 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.443684 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] publish_errors = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.443761 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.443957 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.444046 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.444222 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.444305 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.444477 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.444561 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.444729 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.444816 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.444965 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.445127 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.445211 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.445444 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.445524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] standard_threads = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.445683 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.445764 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] strict_password_check = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.445942 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.446101 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.446180 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.446315 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] use_eventlog = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.446402 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.446527 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] use_journal = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.446636 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.446739 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] use_json = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.446887 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.446963 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] use_stderr = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.447138 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.447224 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] use_syslog = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.447378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.447454 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] watch_log_file = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 22:33:32.447630 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.447770 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.447855 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.448092 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.448175 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.448328 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.448457 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.448562 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.448797 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.449158 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.449326 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.449425 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.449548 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.449758 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.449846 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.450034 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.450110 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.450269 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.450426 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.450512 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.450727 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.450805 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.450966 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.451106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.451191 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.451442 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.451526 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.451677 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.451754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.451917 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.452085 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.452162 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.452381 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.452459 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.452620 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.452742 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.452858 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.453087 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.453166 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.453358 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.453438 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.453599 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.453789 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.453946 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.454061 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.454139 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.454323 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.454476 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.454553 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.454784 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.454868 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.455024 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.455137 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.455264 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.455496 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.455608 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.455850 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.455972 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.456051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.456160 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.456245 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.456463 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.456541 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.456701 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.456826 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.456988 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.457226 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.457443 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] cache.enabled = True {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.457520 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.457617 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.457856 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.457939 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.458099 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.458296 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.458377 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.458561 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.458645 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.458801 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.458920 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.459039 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.459258 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.459336 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.459530 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.459657 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.459774 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.459921 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.460012 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.460251 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.460337 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.460494 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.460586 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.460732 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.460879 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.460955 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.461194 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.461298 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.461452 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.461530 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.461686 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.461867 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.461945 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.462143 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.462308 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.462384 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.462604 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.462683 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.462843 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.462920 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.463105 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.463182 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.463344 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.463423 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.463587 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.463664 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.463825 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.463902 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.464068 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.464147 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.464307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] auth.external = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.464383 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.464544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] auth.mapped = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.464621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.464783 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.464890 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.465028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.465106 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.465269 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.465377 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] auth.password = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.465525 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] auth.token = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.465604 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.465767 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.465845 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.466002 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] catalog.caching = True {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.466078 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.466235 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.466316 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.466470 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.466547 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.466717 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.466794 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.466962 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.467041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.467194 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.467271 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.467418 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] credential.caching = True {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.467499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.467669 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] credential.driver = sql {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.467757 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.467905 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.467989 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.468146 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.468224 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.468387 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.468464 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.468634 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.468716 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.468870 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.468949 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.469135 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.469224 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.469406 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.469486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.469646 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.469724 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.469884 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.469970 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.470126 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.470204 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.470382 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] cache.enabled = True {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.470466 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.470627 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.470716 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.470870 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.470949 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.471117 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.471197 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.471361 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.471469 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] federation.caching = True {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.471597 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.471684 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.471835 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.471913 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] federation.driver = sql {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.472076 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.472159 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.472307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.472393 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.473113 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.473199 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.473382 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.473545 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.473632 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.473812 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.473951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.474070 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.474203 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.474302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.474446 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.474534 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.474689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.474768 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.474933 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.475011 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] identity.caching = True {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.475170 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.475260 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.475408 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.475492 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.475643 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.475722 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.475885 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.475972 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.476122 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.476201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] identity.driver = sql {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.476371 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.476449 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.476608 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.476696 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.476860 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.476941 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.477102 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.477180 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.477361 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.477456 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.477598 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.477677 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.477833 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.477910 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.478080 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.478170 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] auth.external = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.478324 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] auth.mapped = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.478401 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.478571 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.478648 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.478830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.478919 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.479076 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] auth.password = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.479154 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.479315 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] auth.token = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.479400 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.479563 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.479651 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.479800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.479878 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] catalog.caching = True {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.480036 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.480115 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.480275 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.480361 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.480512 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.480589 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.480751 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.480829 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.480990 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.481077 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.481232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.481323 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] credential.caching = True {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.481509 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.481587 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] credential.driver = sql {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.481748 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.481905 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.482006 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.482084 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.482248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.482326 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.482484 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.482563 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.482723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.482800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.482969 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.483046 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.483205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.483285 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.483446 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.483524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.483678 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.password = **** {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.483755 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.483919 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.483998 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.484158 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.484236 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.484402 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.484486 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.484642 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.484722 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.484882 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.484962 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.485133 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.485211 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] federation.caching = True {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.485410 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.485530 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.485652 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.485731 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] federation.driver = sql {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.485886 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.485965 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.486138 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.486227 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.486392 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.486471 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.486633 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.486711 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.486872 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.486951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.487111 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.487189 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.487345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.user = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.487422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.487587 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.487665 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.487843 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.487942 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.488068 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.488149 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] identity.caching = True {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.488914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.489054 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.489172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.489465 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.489655 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.489733 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.489814 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.489930 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] identity.driver = sql {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.490010 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.490168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.490253 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.490412 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.490499 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.490666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.490745 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.490917 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.490995 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.491152 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.491239 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.491387 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.491464 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.491624 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.491703 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.491876 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.491962 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.492128 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.492209 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.492375 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.492459 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.492607 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.492694 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.492857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.492936 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.493097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.493175 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.493359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.493454 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.493601 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.493680 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.493836 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.493915 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.494078 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.494164 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.494317 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.494395 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.494555 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.494633 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.494800 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.494880 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] policy.driver = sql {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.495033 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.495111 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.495280 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.495358 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.495516 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.495595 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.495751 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] receipt.caching = True {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.495829 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.495989 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.496068 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.496235 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.496315 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.496477 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.496555 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.496713 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.496793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.496951 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.497031 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.497191 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.497268 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] resource.caching = True {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.497453 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.password = **** {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.497533 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.497699 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.497780 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] resource.driver = sql {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.497936 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.498013 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.498180 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.498258 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.498423 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.498503 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.498657 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] revoke.caching = True {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.498734 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.498906 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.498984 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.499145 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.499225 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.499379 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.499457 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] role.cache_time = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.499612 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] role.caching = True {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.499689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.499860 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] role.driver = sql {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.499940 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.500092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] role.list_limit = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.500169 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.500338 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.500415 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.500581 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.500661 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.500821 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.500899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.501059 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.user = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.501137 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.501325 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.501408 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.501580 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.501659 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.501816 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.501895 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.502070 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.502151 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.502304 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.502382 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.502554 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.502632 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.502792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.502871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.503027 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.503106 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.503267 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.503346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.503509 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.503686 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.503764 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.503963 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.504197 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.504275 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.504502 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.504585 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.504658 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.504817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.504896 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.505072 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.505159 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.505327 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.505408 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.505576 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.505653 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.505833 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.505916 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.506074 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.506152 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.506312 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.506390 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.506550 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.506637 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.506787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.506866 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.507033 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.507111 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.507288 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.507369 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.507531 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.507610 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.507769 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.507847 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] policy.driver = sql {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.508009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.508089 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.508242 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] token.cache_time = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.508321 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.508479 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] token.caching = True {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.508558 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.508717 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] receipt.caching = True {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.508797 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.508963 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.509095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] token.provider = fernet {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.509201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.509328 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.509475 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.509556 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.509715 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.509793 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.509954 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.510032 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.510197 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] resource.caching = True {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.510277 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.510440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.510518 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.510692 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] resource.driver = sql {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.510770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] trust.driver = sql {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.510930 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.511010 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.511171 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.511249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.511409 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.511487 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.511648 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] revoke.caching = True {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.511727 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.511895 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.511972 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.512126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.512204 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.512368 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] role.cache_time = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.512448 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.512609 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] role.caching = True {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.512687 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.512862 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] role.driver = sql {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.512941 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] database.connection = **** {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.513102 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] role.list_limit = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.513194 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.513382 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.513467 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.513637 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.513715 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.513882 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.513973 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.514121 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.514200 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.514359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.514438 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.514597 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.514684 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.514834 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.514914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.515080 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.515158 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.515315 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.515400 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.515558 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.515636 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.515794 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.515873 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.516042 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.516131 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.516277 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.516355 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.516522 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.516692 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.516771 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.516941 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.517021 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.517198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.517291 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.517390 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93438) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 10 22:33:32.517493 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.517750 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.518006 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.518246 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.518621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.518886 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.519133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.519373 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.519830 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.520076 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.520336 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.520601 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.520858 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.521102 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.521375 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.521614 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] token.cache_time = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.521857 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] token.caching = True {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.522100 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.522355 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] token.provider = fernet {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.522599 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.522862 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.523116 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.523360 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.523610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.523860 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.524114 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] trust.driver = sql {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.524358 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.524602 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.524843 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.525095 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.525374 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.525613 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.525862 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.526166 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.526436 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] database.connection = **** {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.526690 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.526953 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.527198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.527444 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.527684 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.527925 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.528164 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.528403 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.528660 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.528902 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.529149 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.529418 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.529673 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.529916 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.530165 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.530401 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.530646 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.530887 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 22:33:32.531061 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93439) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 10 22:33:32.546737 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=93438) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.546973 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=93438) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.547204 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=93438) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.556364 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=93439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.556598 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=93439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.556824 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=93439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.557161 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=93438) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.567181 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=93439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.587556 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=93438) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.587789 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=93438) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.588788 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=93439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.589027 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=93439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.641741 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=93438) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.642562 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=93439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.693016 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=93438) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.693246 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=93438) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.694239 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=93439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.694481 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=93439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.759750 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.759986 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=93439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.760075 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93438) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.760214 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=93439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.760311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=93438) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.760438 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=93439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.760522 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=93438) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.760660 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=93439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.760743 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=93438) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.760905 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=93439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.760999 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=93438) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.761108 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=93438) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.784353 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.784587 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=93439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.784680 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93438) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.784839 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=93439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.784925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=93438) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.785070 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=93439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.785232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=93438) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.785354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=93439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.785496 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=93438) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.785630 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=93439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.785790 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=93438) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.785908 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.786020 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=93438) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.786255 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93438) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.797074 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=93438) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.797799 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=93439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.810964 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.811205 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=93439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.811461 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=93439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.811562 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93438) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.811684 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=93439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.811767 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=93438) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.811901 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=93439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.811983 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=93438) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.812115 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=93439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.812198 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=93438) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.812334 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=93438) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.812413 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93439) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.812555 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=93438) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.812775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93438) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 22:33:32.829986 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.830199 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 22:33:32.830548 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.830636 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.830757 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 22:33:32.830861 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 22:33:32.831040 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.831124 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.831290 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 22:33:32.831372 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.831559 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.831649 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.831823 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.831903 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.832628 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.832964 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.843791 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.844077 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.844355 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.844438 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.844647 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.844745 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.844921 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.845208 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.853116 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=93438) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:32.853236 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=93439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:32.856221 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=93438) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:32.856318 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=93439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:32.856515 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.856595 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.856805 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.857096 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.857176 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.857414 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.857492 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.857682 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.857932 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.858017 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.858211 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.858282 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.858484 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.858562 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.858761 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.859036 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.859106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.859511 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.860074 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.860467 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.883702 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.885209 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=93438) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:32.885526 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.889825 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.890144 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=93438) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:32.890432 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.890718 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.891343 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=93439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:32.891635 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.897018 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=93439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:32.897254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.897567 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.901195 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=93438) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:32.901520 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.904163 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=93439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:32.904457 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.905645 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=93438) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:32.905944 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.906233 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.906534 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.906831 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.907140 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.907424 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.907701 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.908575 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=93439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:32.908883 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.909182 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.909502 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.910021 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.910119 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.911239 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.911523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.927011 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.927289 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.928391 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.928996 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.931285 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=93438) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:32.932977 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=93439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:32.934436 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.934739 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.936131 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.936436 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.939127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=93438) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:32.940828 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=93439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:32.942952 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=93438) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:32.944687 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=93439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:32.946139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.946445 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.946736 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.947021 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.947301 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.947591 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.947886 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.948194 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.948489 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.948783 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.949347 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.949652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.964528 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=93438) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:32.964820 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.965093 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.965400 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.966648 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=93439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:32.966951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.967227 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.967502 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.972335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.974174 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.974457 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.974683 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.976252 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.976540 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.977268 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=93438) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:32.977580 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.977883 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.978179 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.978465 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.979376 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=93439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:32.979668 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.979957 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.980258 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.980547 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.990482 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=93438) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:32.990768 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.991054 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.991336 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.991622 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.991902 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.992182 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.992744 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=93439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:32.993039 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.993354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.993647 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.993933 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.994216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:32.994508 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.009399 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=93438) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:33.011729 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=93439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:33.012349 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=93438) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:33.014727 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=93439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:33.015312 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.016892 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.017696 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.018414 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=93438) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:33.018707 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.019005 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.019286 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.020892 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=93439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:33.021196 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.021511 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.025602 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.027221 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=93438) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:33.028065 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.029705 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=93439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:33.030195 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.030493 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.030790 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.031071 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.032656 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.032955 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.033248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.033559 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.039292 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=93438) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:33.039603 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.039968 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.041638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=93439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:33.041939 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.042228 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.046859 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=93438) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:33.047148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.047439 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.047718 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.048000 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.048286 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.048566 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.048845 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.049167 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=93439) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 22:33:33.049295 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.049458 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.049549 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.049750 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.049831 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.050036 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.050119 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.050320 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.050422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.050583 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=93438) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.050666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.051381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.051661 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.052059 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.052359 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.052652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.052925 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.053210 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.053507 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=93439) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 22:33:33.085320 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5653b13b99c0 pid: 93438 (default app) Aug 10 22:33:33.087540 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5653b13b99c0 pid: 93439 (default app) Aug 10 22:33:33.088610 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb977a-73a2-48f0-97ce-ffb5eabfeac7 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:33.088901 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb977a-73a2-48f0-97ce-ffb5eabfeac7 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:33.089176 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb977a-73a2-48f0-97ce-ffb5eabfeac7 None None] PATH_INFO: `/v3` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:33.093491 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1/1] 10.222.0.76 () {50 vars in 791 bytes} [Wed Aug 10 22:33:33 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:33:40.739642 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0d87b-54b4-471f-b94b-d19a81d60c66 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:40.740172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0d87b-54b4-471f-b94b-d19a81d60c66 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:40.740172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0d87b-54b4-471f-b94b-d19a81d60c66 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:40.745723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1/2] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:33:40 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:33:40.751011 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ced52-f5ca-439d-973b-829b06e7abbb None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:40.751454 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ced52-f5ca-439d-973b-829b06e7abbb None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:40.751546 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ced52-f5ca-439d-973b-829b06e7abbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:40.822216 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: INFO dbcounter [None req-236ced52-f5ca-439d-973b-829b06e7abbb None None] Registered counter for database keystone Aug 10 22:33:40.841505 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG oslo_db.sqlalchemy.engines [None req-236ced52-f5ca-439d-973b-829b06e7abbb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=93438) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 10 22:33:40.847578 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writer thread running {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 10 22:33:41.032631 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG passlib.handlers.bcrypt [None req-236ced52-f5ca-439d-973b-829b06e7abbb None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=93438) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 10 22:33:41.032952 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG passlib.handlers.bcrypt [None req-236ced52-f5ca-439d-973b-829b06e7abbb None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=93438) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 10 22:33:41.087673 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-236ced52-f5ca-439d-973b-829b06e7abbb None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:33:41.089268 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-236ced52-f5ca-439d-973b-829b06e7abbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:33:41.218645 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 2/3] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:33:40 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 469 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 10 22:33:41.357579 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f0d87b-54b4-471f-b94b-d19a81d60c66 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:33:41.428043 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: INFO dbcounter [None req-89f0d87b-54b4-471f-b94b-d19a81d60c66 None None] Registered counter for database keystone Aug 10 22:33:41.449873 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG oslo_db.sqlalchemy.engines [None req-89f0d87b-54b4-471f-b94b-d19a81d60c66 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=93439) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 10 22:33:41.456331 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writer thread running {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 10 22:33:41.574616 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f0d87b-54b4-471f-b94b-d19a81d60c66 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:33:41.579110 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28b22e-da03-4727-9ee3-8abeb6c0a81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:33:41.579728 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28b22e-da03-4727-9ee3-8abeb6c0a81c admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:41.580010 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28b22e-da03-4727-9ee3-8abeb6c0a81c admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:41.580283 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28b22e-da03-4727-9ee3-8abeb6c0a81c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:41.588943 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb28b22e-da03-4727-9ee3-8abeb6c0a81c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:33:41.642411 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: WARNING keystone.common.rbac_enforcer.enforcer [None req-fb28b22e-da03-4727-9ee3-8abeb6c0a81c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 10 22:33:41.642692 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb28b22e-da03-4727-9ee3-8abeb6c0a81c admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:33:41.652999 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 2/4] 10.222.0.76 () {58 vars in 1324 bytes} [Wed Aug 10 22:33:41 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 22:33:41.658942 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ced52-f5ca-439d-973b-829b06e7abbb None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:33:41.674171 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ced52-f5ca-439d-973b-829b06e7abbb None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:33:41.678662 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5522fb1e-2f8b-4390-8e2f-71e138a572f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:33:41.680348 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522fb1e-2f8b-4390-8e2f-71e138a572f4 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:41.680631 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522fb1e-2f8b-4390-8e2f-71e138a572f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:41.680908 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522fb1e-2f8b-4390-8e2f-71e138a572f4 admin admin] PATH_INFO: `/v3/projects/946936eb2d5946a8b7f7ecfc0535ae77` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:41.682415 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5522fb1e-2f8b-4390-8e2f-71e138a572f4 admin admin] RBAC: Authorizing `identity:get_project(project_id=946936eb2d5946a8b7f7ecfc0535ae77)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:33:41.737037 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5522fb1e-2f8b-4390-8e2f-71e138a572f4 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 10 22:33:41.737400 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5522fb1e-2f8b-4390-8e2f-71e138a572f4 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:33:41.742645 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 3/5] 10.222.0.76 () {56 vars in 1207 bytes} [Wed Aug 10 22:33:41 2022] GET /identity/v3/projects/946936eb2d5946a8b7f7ecfc0535ae77 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:33:43.813538 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9703ba-494a-47a9-860c-f14083ef66fc None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:43.814051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9703ba-494a-47a9-860c-f14083ef66fc None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:43.814051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9703ba-494a-47a9-860c-f14083ef66fc None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:43.818085 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 3/6] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:33:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:33:43.824635 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0d87b-9478-4c75-909e-5307992508f2 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:43.825062 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0d87b-9478-4c75-909e-5307992508f2 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:43.825184 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0d87b-9478-4c75-909e-5307992508f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:43.846757 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-4cc0d87b-9478-4c75-909e-5307992508f2 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:33:43.848276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-4cc0d87b-9478-4c75-909e-5307992508f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:33:43.856708 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 4/7] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:33:43 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 10 22:33:43.879427 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9703ba-494a-47a9-860c-f14083ef66fc None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:33:43.895857 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9703ba-494a-47a9-860c-f14083ef66fc None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:33:43.902902 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d8d122-396d-4a0d-b449-4b1460b1eae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:33:43.903941 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8d122-396d-4a0d-b449-4b1460b1eae0 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:43.904456 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8d122-396d-4a0d-b449-4b1460b1eae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:43.904977 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8d122-396d-4a0d-b449-4b1460b1eae0 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:43.916178 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d8d122-396d-4a0d-b449-4b1460b1eae0 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:33:43.919655 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d8d122-396d-4a0d-b449-4b1460b1eae0 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:33:43.937240 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 4/8] 10.222.0.76 () {58 vars in 1324 bytes} [Wed Aug 10 22:33:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 22:33:43.942826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc0d87b-9478-4c75-909e-5307992508f2 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:33:43.952485 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc0d87b-9478-4c75-909e-5307992508f2 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:33:43.957072 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b4b0f-6d2b-4f85-986a-abfcd9fdedb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:33:43.957726 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b4b0f-6d2b-4f85-986a-abfcd9fdedb9 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:43.958006 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b4b0f-6d2b-4f85-986a-abfcd9fdedb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:43.958281 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b4b0f-6d2b-4f85-986a-abfcd9fdedb9 admin admin] PATH_INFO: `/v3/users/d91ed101f8354003baf64fb2ad037357` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:43.959596 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b4b0f-6d2b-4f85-986a-abfcd9fdedb9 admin admin] RBAC: Authorizing `identity:get_user(user_id=d91ed101f8354003baf64fb2ad037357)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:33:43.962089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b4b0f-6d2b-4f85-986a-abfcd9fdedb9 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:33:43.963620 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 5/9] 10.222.0.76 () {56 vars in 1198 bytes} [Wed Aug 10 22:33:43 2022] GET /identity/v3/users/d91ed101f8354003baf64fb2ad037357 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:33:46.081152 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10dc21-6aab-4979-9aab-3559b7cc97be None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:46.081729 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10dc21-6aab-4979-9aab-3559b7cc97be None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:46.081729 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10dc21-6aab-4979-9aab-3559b7cc97be None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:46.083141 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 5/10] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:33:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:33:46.084875 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4dae5-efa4-4fb6-ab3b-31d606734af7 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:46.085319 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4dae5-efa4-4fb6-ab3b-31d606734af7 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:46.085481 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4dae5-efa4-4fb6-ab3b-31d606734af7 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:46.088022 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 6/11] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:33:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:33:46.088950 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da239f0-4df7-4b42-b11b-fc25d4aa40ca None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:46.089243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da239f0-4df7-4b42-b11b-fc25d4aa40ca None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:46.089554 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da239f0-4df7-4b42-b11b-fc25d4aa40ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:46.093176 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434bc48-9c2c-4410-b859-3e2bf3c27b06 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:46.093496 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434bc48-9c2c-4410-b859-3e2bf3c27b06 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:46.093773 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434bc48-9c2c-4410-b859-3e2bf3c27b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:46.114365 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-4434bc48-9c2c-4410-b859-3e2bf3c27b06 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:33:46.117204 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-4434bc48-9c2c-4410-b859-3e2bf3c27b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:33:46.138627 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG passlib.handlers.bcrypt [None req-1da239f0-4df7-4b42-b11b-fc25d4aa40ca None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=93439) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 10 22:33:46.139060 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG passlib.handlers.bcrypt [None req-1da239f0-4df7-4b42-b11b-fc25d4aa40ca None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=93439) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 10 22:33:46.184758 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-1da239f0-4df7-4b42-b11b-fc25d4aa40ca None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:33:46.186407 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-1da239f0-4df7-4b42-b11b-fc25d4aa40ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:33:46.197950 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 7/12] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:33:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:33:46.220615 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4434bc48-9c2c-4410-b859-3e2bf3c27b06 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:33:46.245445 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:33:46.288496 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 6/13] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:33:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:33:46.312452 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da239f0-4df7-4b42-b11b-fc25d4aa40ca None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:33:46.328165 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4434bc48-9c2c-4410-b859-3e2bf3c27b06 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:33:46.362200 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1570a109-6158-4aee-9cbd-34f7da21b49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:33:46.362818 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570a109-6158-4aee-9cbd-34f7da21b49e None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:46.363101 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570a109-6158-4aee-9cbd-34f7da21b49e None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:46.363376 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570a109-6158-4aee-9cbd-34f7da21b49e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:46.424276 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da239f0-4df7-4b42-b11b-fc25d4aa40ca None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:33:46.427488 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:33:46.436739 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1570a109-6158-4aee-9cbd-34f7da21b49e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:33:46.438244 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1570a109-6158-4aee-9cbd-34f7da21b49e None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:33:46.462921 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:33:46.464560 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71322e78-f62f-4011-bbdf-a758f357a7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:33:46.465738 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71322e78-f62f-4011-bbdf-a758f357a7ff None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:46.466022 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71322e78-f62f-4011-bbdf-a758f357a7ff None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:46.466300 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71322e78-f62f-4011-bbdf-a758f357a7ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:46.479253 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71322e78-f62f-4011-bbdf-a758f357a7ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:33:46.487717 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-71322e78-f62f-4011-bbdf-a758f357a7ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:33:46.487717 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:33:46.487717 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:33:46.487717 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:33:46.487717 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:33:46.487717 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:33:46.487717 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:33:46.487717 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:33:46.487717 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:33:46.487717 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:33:46.487717 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:33:46.487717 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:33:46.487717 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:33:46.487717 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:33:46.487717 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:33:46.487717 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:33:46.488642 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:33:46.488642 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:33:46.488642 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:33:46.488642 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:33:46.488642 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:33:46.488642 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:33:46.488642 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:33:46.488642 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:33:46.488642 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:33:46.488642 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:33:46.488642 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:33:46.488642 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:33:46.488642 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:33:46.488642 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:33:46.488642 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:33:46.488642 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:33:46.489486 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:33:46.489486 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:33:46.489486 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:33:46.489486 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:33:46.489486 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:33:46.489486 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:33:46.489486 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:33:46.489486 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:33:46.489486 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:33:46.489486 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:33:46.489486 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:33:46.489486 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:33:46.489486 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 7/14] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:33:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:33:46.495505 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71322e78-f62f-4011-bbdf-a758f357a7ff None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:33:46.543621 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 8/15] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:33:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 22:33:46.549223 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1570a109-6158-4aee-9cbd-34f7da21b49e None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:33:46.597913 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71322e78-f62f-4011-bbdf-a758f357a7ff None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:33:46.636628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246e6b87-eb2f-4040-b8b6-188d95f71249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:33:46.637161 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e6b87-eb2f-4040-b8b6-188d95f71249 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:46.637478 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e6b87-eb2f-4040-b8b6-188d95f71249 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:46.637755 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e6b87-eb2f-4040-b8b6-188d95f71249 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:46.638447 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246e6b87-eb2f-4040-b8b6-188d95f71249 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:33:46.638826 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246e6b87-eb2f-4040-b8b6-188d95f71249 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:33:46.640333 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246e6b87-eb2f-4040-b8b6-188d95f71249 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:33:46.656629 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 8/16] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:33:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:33:46.662400 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246e6b87-eb2f-4040-b8b6-188d95f71249 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:33:46.698564 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:33:46.765239 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1570a109-6158-4aee-9cbd-34f7da21b49e None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:33:46.789075 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:33:46.796095 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246e6b87-eb2f-4040-b8b6-188d95f71249 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:33:46.801508 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebadf4bd-1d6d-4c7f-8b02-2eb4b2edf47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:33:46.802150 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebadf4bd-1d6d-4c7f-8b02-2eb4b2edf47a None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:46.802433 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebadf4bd-1d6d-4c7f-8b02-2eb4b2edf47a None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:46.802709 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebadf4bd-1d6d-4c7f-8b02-2eb4b2edf47a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:46.813967 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebadf4bd-1d6d-4c7f-8b02-2eb4b2edf47a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:33:46.815447 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebadf4bd-1d6d-4c7f-8b02-2eb4b2edf47a None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:33:46.831207 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981a74e-516b-480a-9468-b43c5d4d33df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:33:46.831843 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981a74e-516b-480a-9468-b43c5d4d33df None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:46.832126 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981a74e-516b-480a-9468-b43c5d4d33df None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:46.832405 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981a74e-516b-480a-9468-b43c5d4d33df None admin] PATH_INFO: `/v3/users/d91ed101f8354003baf64fb2ad037357` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:46.833946 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8981a74e-516b-480a-9468-b43c5d4d33df None admin] RBAC: Authorizing `identity:get_user(user_id=d91ed101f8354003baf64fb2ad037357)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:33:46.834559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-ebadf4bd-1d6d-4c7f-8b02-2eb4b2edf47a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:33:46.834559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:33:46.834559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:33:46.834559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:33:46.834559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:33:46.834559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:33:46.834559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:33:46.834559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:33:46.834559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:33:46.834559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:33:46.834559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:33:46.834559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:33:46.834559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:33:46.834559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:33:46.834559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:33:46.834559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:33:46.835376 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:33:46.835376 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:33:46.835376 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:33:46.835376 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:33:46.835376 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:33:46.835376 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:33:46.835376 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:33:46.835376 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:33:46.835376 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:33:46.835376 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:33:46.835376 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:33:46.835376 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:33:46.835376 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:33:46.835376 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:33:46.835376 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:33:46.835376 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:33:46.835376 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:33:46.836162 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:33:46.836162 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:33:46.836162 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:33:46.836337 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8981a74e-516b-480a-9468-b43c5d4d33df None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:33:46.836407 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 9/17] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:33:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:33:46.837936 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 9/18] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:33:46 2022] GET /identity/v3/users/d91ed101f8354003baf64fb2ad037357 => generated 247 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:33:46.842620 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebadf4bd-1d6d-4c7f-8b02-2eb4b2edf47a None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:33:46.842921 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981a74e-516b-480a-9468-b43c5d4d33df None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:33:46.935999 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:33:46.943527 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981a74e-516b-480a-9468-b43c5d4d33df None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:33:46.953117 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebadf4bd-1d6d-4c7f-8b02-2eb4b2edf47a None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:33:46.983036 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689544ba-64a4-4430-a8e9-46e975915568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:33:46.983676 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689544ba-64a4-4430-a8e9-46e975915568 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:46.983955 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689544ba-64a4-4430-a8e9-46e975915568 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:46.984228 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689544ba-64a4-4430-a8e9-46e975915568 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:46.985383 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689544ba-64a4-4430-a8e9-46e975915568 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:33:46.986855 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689544ba-64a4-4430-a8e9-46e975915568 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:33:46.988415 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 10/19] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:33:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:33:46.991509 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7256bc7-862b-4441-a70a-ec35d637c8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:33:46.992128 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7256bc7-862b-4441-a70a-ec35d637c8d4 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:46.992412 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7256bc7-862b-4441-a70a-ec35d637c8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:46.992686 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7256bc7-862b-4441-a70a-ec35d637c8d4 None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:46.993368 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7256bc7-862b-4441-a70a-ec35d637c8d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:33:46.994275 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7256bc7-862b-4441-a70a-ec35d637c8d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:33:46.994693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689544ba-64a4-4430-a8e9-46e975915568 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:33:46.995988 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7256bc7-862b-4441-a70a-ec35d637c8d4 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:33:47.014544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 10/20] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:33:47.111142 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689544ba-64a4-4430-a8e9-46e975915568 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:33:47.147009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3a536-074e-4ea3-b668-556f55196520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:33:47.147631 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3a536-074e-4ea3-b668-556f55196520 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:47.147913 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3a536-074e-4ea3-b668-556f55196520 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:47.148189 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3a536-074e-4ea3-b668-556f55196520 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:47.148929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d3a536-074e-4ea3-b668-556f55196520 None admin] RBAC: Adding query filter params (role.id=6641e95fdc1f4c098ebc372bb692d227, scope.domain.id=default, user.id=d91ed101f8354003baf64fb2ad037357) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:33:47.149238 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d3a536-074e-4ea3-b668-556f55196520 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:33:47.151396 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d3a536-074e-4ea3-b668-556f55196520 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:33:47.160958 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 11/21] 10.222.0.76 () {56 vars in 1322 bytes} [Wed Aug 10 22:33:46 2022] GET /identity/v3/role_assignments?user.id=d91ed101f8354003baf64fb2ad037357&scope.domain.id=default&role.id=6641e95fdc1f4c098ebc372bb692d227 => generated 233 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:33:49.502242 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878faad-3436-4d5d-8bfd-91cf2179c2f1 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:49.502737 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878faad-3436-4d5d-8bfd-91cf2179c2f1 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:49.502808 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878faad-3436-4d5d-8bfd-91cf2179c2f1 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:49.504109 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 11/22] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:33:49.509551 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88008f3b-da8b-49a6-b4e8-d953f705706c None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:49.510486 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88008f3b-da8b-49a6-b4e8-d953f705706c None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:49.510965 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88008f3b-da8b-49a6-b4e8-d953f705706c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:49.535580 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-88008f3b-da8b-49a6-b4e8-d953f705706c None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:33:49.537095 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-88008f3b-da8b-49a6-b4e8-d953f705706c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:33:49.574648 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:33:49.648039 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 12/23] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:33:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:33:49.691833 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878faad-3436-4d5d-8bfd-91cf2179c2f1 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:33:49.744376 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:33:49.782866 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e440d3-77cf-4dcf-8665-323eebba1eec None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:49.783480 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e440d3-77cf-4dcf-8665-323eebba1eec None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:49.783988 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e440d3-77cf-4dcf-8665-323eebba1eec None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:49.785573 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 13/24] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:33:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:33:49.790940 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4108f8-197e-41f6-9746-a1154b7150f4 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:49.792305 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4108f8-197e-41f6-9746-a1154b7150f4 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:49.792840 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4108f8-197e-41f6-9746-a1154b7150f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:49.811945 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-ca4108f8-197e-41f6-9746-a1154b7150f4 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:33:49.813639 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-ca4108f8-197e-41f6-9746-a1154b7150f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:33:49.847540 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878faad-3436-4d5d-8bfd-91cf2179c2f1 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:33:49.908785 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d55f24-d466-411e-a0a3-704ede757cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:33:49.909472 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d55f24-d466-411e-a0a3-704ede757cd7 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:49.909757 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d55f24-d466-411e-a0a3-704ede757cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:49.910098 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d55f24-d466-411e-a0a3-704ede757cd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:49.931753 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d55f24-d466-411e-a0a3-704ede757cd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:33:49.933778 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d55f24-d466-411e-a0a3-704ede757cd7 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:33:49.966081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-62d55f24-d466-411e-a0a3-704ede757cd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:33:49.966081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:33:49.966081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:33:49.966081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:33:49.966081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:33:49.966081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:33:49.966081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:33:49.966081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:33:49.966081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:33:49.966081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:33:49.966081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:33:49.966081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:33:49.966081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:33:49.966081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:33:49.966081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:33:49.966081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:33:49.966959 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:33:49.966959 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:33:49.966959 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:33:49.966959 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:33:49.966959 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:33:49.966959 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:33:49.966959 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:33:49.966959 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:33:49.966959 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:33:49.966959 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:33:49.966959 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:33:49.966959 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:33:49.966959 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:33:49.966959 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:33:49.966959 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:33:49.966959 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:33:49.966959 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:33:49.967817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:33:49.967817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:33:49.967817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:33:49.969625 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 12/25] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:33:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:33:49.969948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 14/26] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:33:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:33:49.971186 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02add7-0af5-4f40-8ea5-af1ca7364853 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:49.971623 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02add7-0af5-4f40-8ea5-af1ca7364853 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:49.972031 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02add7-0af5-4f40-8ea5-af1ca7364853 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:49.973451 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 13/27] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:33:49.979240 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5b9a2-4b8c-42a7-99aa-d53ba7b2405b None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:49.979792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5b9a2-4b8c-42a7-99aa-d53ba7b2405b None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:49.980291 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5b9a2-4b8c-42a7-99aa-d53ba7b2405b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:49.997429 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02add7-0af5-4f40-8ea5-af1ca7364853 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:33:50.018448 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-27f5b9a2-4b8c-42a7-99aa-d53ba7b2405b None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:33:50.020122 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-27f5b9a2-4b8c-42a7-99aa-d53ba7b2405b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:33:50.119313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:33:50.138765 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02add7-0af5-4f40-8ea5-af1ca7364853 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:33:50.177478 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e668319-73c7-400a-b44b-812bbf124f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:33:50.178335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e668319-73c7-400a-b44b-812bbf124f40 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:50.178812 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e668319-73c7-400a-b44b-812bbf124f40 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:50.179223 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e668319-73c7-400a-b44b-812bbf124f40 None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:50.180096 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e668319-73c7-400a-b44b-812bbf124f40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:33:50.180590 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e668319-73c7-400a-b44b-812bbf124f40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:33:50.182940 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e668319-73c7-400a-b44b-812bbf124f40 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:33:50.196274 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 14/28] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:33:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:33:50.197367 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e668319-73c7-400a-b44b-812bbf124f40 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:33:50.263532 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 15/29] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:33:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:33:50.265235 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a043a-fd34-4c35-bbe4-6471741e24dc None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:50.265614 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a043a-fd34-4c35-bbe4-6471741e24dc None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:50.266391 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a043a-fd34-4c35-bbe4-6471741e24dc None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:50.268210 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 16/30] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:33:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:33:50.272246 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a043a-fd34-4c35-bbe4-6471741e24dc None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:33:50.294325 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:33:50.324865 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e668319-73c7-400a-b44b-812bbf124f40 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:33:50.374310 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc049bbe-0557-445a-a69f-2de1f5cc95ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:33:50.375069 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc049bbe-0557-445a-a69f-2de1f5cc95ca None admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:50.375424 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc049bbe-0557-445a-a69f-2de1f5cc95ca None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:50.375781 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc049bbe-0557-445a-a69f-2de1f5cc95ca None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:50.376801 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc049bbe-0557-445a-a69f-2de1f5cc95ca None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:33:50.378779 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc049bbe-0557-445a-a69f-2de1f5cc95ca None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:33:50.404046 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a043a-fd34-4c35-bbe4-6471741e24dc None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:33:50.455852 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf6558-99c6-4fc7-ad26-a1f8b44ad0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:33:50.456579 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf6558-99c6-4fc7-ad26-a1f8b44ad0a3 None admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:50.456936 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf6558-99c6-4fc7-ad26-a1f8b44ad0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:50.457301 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf6558-99c6-4fc7-ad26-a1f8b44ad0a3 None admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:50.458038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf6558-99c6-4fc7-ad26-a1f8b44ad0a3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:33:50.459573 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf6558-99c6-4fc7-ad26-a1f8b44ad0a3 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:33:50.535398 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 15/31] 10.222.0.76 () {60 vars in 1131 bytes} [Wed Aug 10 22:33:50 2022] POST /identity/v3/roles => generated 225 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:33:50.537606 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14676077-6874-4579-b5ec-4d8b6a1022e5 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:50.538099 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14676077-6874-4579-b5ec-4d8b6a1022e5 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:50.538799 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14676077-6874-4579-b5ec-4d8b6a1022e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:50.554692 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:33:50.571817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-14676077-6874-4579-b5ec-4d8b6a1022e5 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:33:50.574078 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-14676077-6874-4579-b5ec-4d8b6a1022e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:33:50.704471 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 17/32] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 10 22:33:50 2022] POST /identity/v3/projects => generated 301 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:33:50.711248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf6558-99c6-4fc7-ad26-a1f8b44ad0a3 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:33:50.730104 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 16/33] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:33:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:33:50.767574 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14676077-6874-4579-b5ec-4d8b6a1022e5 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:33:50.818778 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:33:50.865671 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf6558-99c6-4fc7-ad26-a1f8b44ad0a3 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:33:50.908117 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c075e-3782-4f95-a1a4-ba763b4ee2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:33:50.908757 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c075e-3782-4f95-a1a4-ba763b4ee2f5 None admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:50.909044 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c075e-3782-4f95-a1a4-ba763b4ee2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:50.909378 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c075e-3782-4f95-a1a4-ba763b4ee2f5 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:50.910131 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c075e-3782-4f95-a1a4-ba763b4ee2f5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:33:50.911687 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c075e-3782-4f95-a1a4-ba763b4ee2f5 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:33:50.933025 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 18/34] 10.222.0.76 () {60 vars in 1131 bytes} [Wed Aug 10 22:33:50 2022] POST /identity/v3/roles => generated 219 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:33:50.934838 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ae18a-348e-4cc9-ae65-dd18e4da68ee None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:50.934967 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ae18a-348e-4cc9-ae65-dd18e4da68ee None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:50.935143 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ae18a-348e-4cc9-ae65-dd18e4da68ee None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:50.936673 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 19/35] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:33:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:33:50.942501 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c5ed6-d049-48d0-ac55-45202df959ea None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:50.942832 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c5ed6-d049-48d0-ac55-45202df959ea None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:50.943073 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c5ed6-d049-48d0-ac55-45202df959ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:50.961971 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14676077-6874-4579-b5ec-4d8b6a1022e5 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:33:50.980309 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-aa2c5ed6-d049-48d0-ac55-45202df959ea None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:33:50.983801 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-aa2c5ed6-d049-48d0-ac55-45202df959ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:33:51.051829 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a703bd-4e48-401a-94fd-3c960a2a0f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:33:51.053160 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a703bd-4e48-401a-94fd-3c960a2a0f29 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:51.054241 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a703bd-4e48-401a-94fd-3c960a2a0f29 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:51.054738 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a703bd-4e48-401a-94fd-3c960a2a0f29 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:51.061741 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:33:51.081300 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a703bd-4e48-401a-94fd-3c960a2a0f29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:33:51.087625 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-89a703bd-4e48-401a-94fd-3c960a2a0f29 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:33:51.087625 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:33:51.087625 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:33:51.087625 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:33:51.087625 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:33:51.087625 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:33:51.087625 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:33:51.087625 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:33:51.087625 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:33:51.087625 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:33:51.087625 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:33:51.087625 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:33:51.087625 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:33:51.087625 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:33:51.087625 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:33:51.087625 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:33:51.088517 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:33:51.088517 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:33:51.088517 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:33:51.088517 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:33:51.088517 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:33:51.088517 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:33:51.088517 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:33:51.088517 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:33:51.088517 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:33:51.088517 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:33:51.088517 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:33:51.088517 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:33:51.088517 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:33:51.088517 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:33:51.088517 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:33:51.088517 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:33:51.089899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:33:51.089899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:33:51.089899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:33:51.089899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:33:51.089899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:33:51.089899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:33:51.089899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:33:51.089899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:33:51.089899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:33:51.089899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:33:51.089899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:33:51.089899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:33:51.096060 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 20/36] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:33:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:33:51.096152 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 17/37] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:33:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:33:51.096219 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada01ba5-b6fc-456d-970b-098e305950c0 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:51.096219 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada01ba5-b6fc-456d-970b-098e305950c0 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:51.096219 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada01ba5-b6fc-456d-970b-098e305950c0 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:51.096219 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 21/38] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:33:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:33:51.099882 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a703bd-4e48-401a-94fd-3c960a2a0f29 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:33:51.100350 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3896852-2eef-48c9-ae54-d673e2fad7fb None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:51.100735 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3896852-2eef-48c9-ae54-d673e2fad7fb None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:51.101024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3896852-2eef-48c9-ae54-d673e2fad7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:51.121973 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-a3896852-2eef-48c9-ae54-d673e2fad7fb None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:33:51.123381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-a3896852-2eef-48c9-ae54-d673e2fad7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:33:51.182663 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:33:51.199069 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 22/39] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:33:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:33:51.205670 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3896852-2eef-48c9-ae54-d673e2fad7fb None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:33:51.230730 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a703bd-4e48-401a-94fd-3c960a2a0f29 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:33:51.264833 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306725ac-18c3-42cd-9c70-5d036fb0fb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:33:51.265486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306725ac-18c3-42cd-9c70-5d036fb0fb75 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:51.265774 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306725ac-18c3-42cd-9c70-5d036fb0fb75 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:51.266051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306725ac-18c3-42cd-9c70-5d036fb0fb75 None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:51.266722 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306725ac-18c3-42cd-9c70-5d036fb0fb75 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:33:51.267027 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306725ac-18c3-42cd-9c70-5d036fb0fb75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:33:51.268532 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306725ac-18c3-42cd-9c70-5d036fb0fb75 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:33:51.280981 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 18/40] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:33:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:33:51.281807 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306725ac-18c3-42cd-9c70-5d036fb0fb75 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:33:51.302640 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3896852-2eef-48c9-ae54-d673e2fad7fb None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:33:51.337396 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d3862-41a1-43b6-8d88-8f2face7ed87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:33:51.338030 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d3862-41a1-43b6-8d88-8f2face7ed87 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:51.338312 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d3862-41a1-43b6-8d88-8f2face7ed87 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:51.338591 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d3862-41a1-43b6-8d88-8f2face7ed87 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:51.340235 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:33:51.340338 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4d3862-41a1-43b6-8d88-8f2face7ed87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:33:51.341827 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4d3862-41a1-43b6-8d88-8f2face7ed87 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:33:51.343430 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 23/41] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:33:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:33:51.344182 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d3862-41a1-43b6-8d88-8f2face7ed87 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:33:51.384881 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306725ac-18c3-42cd-9c70-5d036fb0fb75 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:33:51.400202 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:33:51.420352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26401e3d-2eb7-4e91-9ebf-15eb5a713c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:33:51.420977 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26401e3d-2eb7-4e91-9ebf-15eb5a713c89 None admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:51.421272 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26401e3d-2eb7-4e91-9ebf-15eb5a713c89 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:51.421581 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26401e3d-2eb7-4e91-9ebf-15eb5a713c89 None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:51.422248 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26401e3d-2eb7-4e91-9ebf-15eb5a713c89 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:33:51.423649 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26401e3d-2eb7-4e91-9ebf-15eb5a713c89 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:33:51.438458 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 19/42] 10.222.0.76 () {60 vars in 1131 bytes} [Wed Aug 10 22:33:51 2022] POST /identity/v3/roles => generated 223 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:33:51.449662 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26401e3d-2eb7-4e91-9ebf-15eb5a713c89 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:33:51.449753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d3862-41a1-43b6-8d88-8f2face7ed87 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:33:51.488147 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb396772-4c94-4f21-8e69-a38549a35a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:33:51.488808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb396772-4c94-4f21-8e69-a38549a35a00 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:51.489096 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb396772-4c94-4f21-8e69-a38549a35a00 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:51.489409 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb396772-4c94-4f21-8e69-a38549a35a00 None admin] PATH_INFO: `/v3/users/d91ed101f8354003baf64fb2ad037357` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:51.490630 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb396772-4c94-4f21-8e69-a38549a35a00 None admin] RBAC: Authorizing `identity:get_user(user_id=d91ed101f8354003baf64fb2ad037357)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:33:51.492141 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb396772-4c94-4f21-8e69-a38549a35a00 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:33:51.494085 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 24/43] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:33:51 2022] GET /identity/v3/users/d91ed101f8354003baf64fb2ad037357 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:33:51.500181 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb396772-4c94-4f21-8e69-a38549a35a00 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:33:51.572263 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26401e3d-2eb7-4e91-9ebf-15eb5a713c89 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:33:51.593684 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:33:51.615424 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d650896-7eba-4d44-9d69-b3896eff396b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:33:51.615649 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb396772-4c94-4f21-8e69-a38549a35a00 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:33:51.616050 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d650896-7eba-4d44-9d69-b3896eff396b None admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:51.616333 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d650896-7eba-4d44-9d69-b3896eff396b None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:51.616608 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d650896-7eba-4d44-9d69-b3896eff396b None admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:51.617257 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d650896-7eba-4d44-9d69-b3896eff396b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:33:51.621320 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d650896-7eba-4d44-9d69-b3896eff396b None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:33:51.643940 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:33:51.654326 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 20/44] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 10 22:33:51 2022] POST /identity/v3/projects => generated 312 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:33:51.660635 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3a41b-ea28-4721-8d57-ec3c4b5a9a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:33:51.661378 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3a41b-ea28-4721-8d57-ec3c4b5a9a02 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:51.661668 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3a41b-ea28-4721-8d57-ec3c4b5a9a02 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:51.661941 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3a41b-ea28-4721-8d57-ec3c4b5a9a02 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:51.663131 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da3a41b-ea28-4721-8d57-ec3c4b5a9a02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:33:51.664650 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da3a41b-ea28-4721-8d57-ec3c4b5a9a02 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:33:51.666365 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 25/45] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:33:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:33:51.672822 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d650896-7eba-4d44-9d69-b3896eff396b None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:33:51.781560 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d650896-7eba-4d44-9d69-b3896eff396b None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:33:51.818412 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881c07dc-0522-4764-b808-c9e2e62ffc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:33:51.819083 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c07dc-0522-4764-b808-c9e2e62ffc10 None admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:51.819370 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c07dc-0522-4764-b808-c9e2e62ffc10 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:51.819652 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c07dc-0522-4764-b808-c9e2e62ffc10 None admin] PATH_INFO: `/v3/domains/default/users/d91ed101f8354003baf64fb2ad037357/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:51.821684 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881c07dc-0522-4764-b808-c9e2e62ffc10 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d91ed101f8354003baf64fb2ad037357, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:33:51.823286 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881c07dc-0522-4764-b808-c9e2e62ffc10 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:33:51.837364 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 21/46] 10.222.0.76 () {58 vars in 1361 bytes} [Wed Aug 10 22:33:51 2022] PUT /identity/v3/domains/default/users/d91ed101f8354003baf64fb2ad037357/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:33:53.678541 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45e932-2368-4632-a35e-9e57bc128f27 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:53.679050 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45e932-2368-4632-a35e-9e57bc128f27 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:53.679131 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45e932-2368-4632-a35e-9e57bc128f27 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:53.680541 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 26/47] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:33:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:33:53.686288 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281076b5-a01f-41dd-8ceb-4d7b1255fd8d None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:53.686715 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281076b5-a01f-41dd-8ceb-4d7b1255fd8d None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:53.686966 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281076b5-a01f-41dd-8ceb-4d7b1255fd8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:53.704766 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-281076b5-a01f-41dd-8ceb-4d7b1255fd8d None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:33:53.705970 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-281076b5-a01f-41dd-8ceb-4d7b1255fd8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:33:53.729136 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:33:53.773850 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 22/48] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:33:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:33:53.798112 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a45e932-2368-4632-a35e-9e57bc128f27 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:33:53.897706 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c228fc-cee7-4020-86b1-f8602e878216 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:53.897893 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a45e932-2368-4632-a35e-9e57bc128f27 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:33:53.897979 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c228fc-cee7-4020-86b1-f8602e878216 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:53.898272 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c228fc-cee7-4020-86b1-f8602e878216 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:53.899623 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 23/49] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:33:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:33:53.905165 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8baba-3658-4346-bfcc-454938066eae None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:53.905483 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8baba-3658-4346-bfcc-454938066eae None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:53.905767 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8baba-3658-4346-bfcc-454938066eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:53.924153 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-1fd8baba-3658-4346-bfcc-454938066eae None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:33:53.925394 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-1fd8baba-3658-4346-bfcc-454938066eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:33:53.935041 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0dd3dc-2636-4fd1-a177-b068ac1e480b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:33:53.935618 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0dd3dc-2636-4fd1-a177-b068ac1e480b None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:53.935930 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0dd3dc-2636-4fd1-a177-b068ac1e480b None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:53.936182 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0dd3dc-2636-4fd1-a177-b068ac1e480b None admin] PATH_INFO: `/v3/domains/default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:53.937328 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0dd3dc-2636-4fd1-a177-b068ac1e480b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:33:53.938790 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0dd3dc-2636-4fd1-a177-b068ac1e480b None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:33:53.940364 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 27/50] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:33:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:33:53.945983 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0dd3dc-2636-4fd1-a177-b068ac1e480b None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:33:53.997196 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 24/51] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:33:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:33:54.007451 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:33:54.019443 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8baba-3658-4346-bfcc-454938066eae None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:33:54.050263 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0dd3dc-2636-4fd1-a177-b068ac1e480b None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:33:54.085662 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418c711-ccf4-413c-8431-45a1f7e885e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:33:54.086273 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418c711-ccf4-413c-8431-45a1f7e885e5 None admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:54.086558 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418c711-ccf4-413c-8431-45a1f7e885e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:54.086835 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418c711-ccf4-413c-8431-45a1f7e885e5 None admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:54.087481 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418c711-ccf4-413c-8431-45a1f7e885e5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:33:54.088929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418c711-ccf4-413c-8431-45a1f7e885e5 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:33:54.113104 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:33:54.116981 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 28/52] 10.222.0.76 () {60 vars in 1140 bytes} [Wed Aug 10 22:33:53 2022] POST /identity/v3/projects => generated 298 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:33:54.123677 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8baba-3658-4346-bfcc-454938066eae None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:33:54.163335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0243948-0e80-4420-bedb-8c0ca3e9c897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:33:54.164008 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0243948-0e80-4420-bedb-8c0ca3e9c897 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:54.164294 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0243948-0e80-4420-bedb-8c0ca3e9c897 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:54.164573 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0243948-0e80-4420-bedb-8c0ca3e9c897 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:54.177058 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0243948-0e80-4420-bedb-8c0ca3e9c897 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:33:54.181968 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-c0243948-0e80-4420-bedb-8c0ca3e9c897 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:33:54.181968 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:33:54.181968 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:33:54.181968 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:33:54.181968 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:33:54.181968 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:33:54.181968 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:33:54.181968 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:33:54.181968 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:33:54.181968 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:33:54.181968 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:33:54.181968 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:33:54.181968 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:33:54.181968 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:33:54.181968 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:33:54.181968 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:33:54.183051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:33:54.183051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:33:54.183051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:33:54.183051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:33:54.183051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:33:54.183051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:33:54.183051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:33:54.183051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:33:54.183051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:33:54.183051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:33:54.183051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:33:54.183051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:33:54.183051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:33:54.183051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:33:54.183051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:33:54.183051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:33:54.184269 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:33:54.184269 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:33:54.184269 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:33:54.184269 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:33:54.184269 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:33:54.184269 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:33:54.184269 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:33:54.184269 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:33:54.184269 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:33:54.184269 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:33:54.184269 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:33:54.184269 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:33:54.184269 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 25/53] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:33:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:33:54.189820 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418c711-ccf4-413c-8431-45a1f7e885e5 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:33:54.299718 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418c711-ccf4-413c-8431-45a1f7e885e5 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:33:54.313337 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:33:54.336649 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075be853-ea5c-4357-bd69-4c16473d25b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:33:54.337295 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075be853-ea5c-4357-bd69-4c16473d25b6 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:54.337582 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075be853-ea5c-4357-bd69-4c16473d25b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:54.337860 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075be853-ea5c-4357-bd69-4c16473d25b6 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:54.338565 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075be853-ea5c-4357-bd69-4c16473d25b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:33:54.338861 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075be853-ea5c-4357-bd69-4c16473d25b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:33:54.340678 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075be853-ea5c-4357-bd69-4c16473d25b6 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:33:54.349207 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 29/54] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:33:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:33:54.354673 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0243948-0e80-4420-bedb-8c0ca3e9c897 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:33:54.459583 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0243948-0e80-4420-bedb-8c0ca3e9c897 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:33:54.493388 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102da71e-8bc4-4498-9876-2e1b75a1144a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:33:54.494018 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102da71e-8bc4-4498-9876-2e1b75a1144a None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:54.494300 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102da71e-8bc4-4498-9876-2e1b75a1144a None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:54.494573 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102da71e-8bc4-4498-9876-2e1b75a1144a None admin] PATH_INFO: `/v3/users/d91ed101f8354003baf64fb2ad037357` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:54.495713 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102da71e-8bc4-4498-9876-2e1b75a1144a None admin] RBAC: Authorizing `identity:get_user(user_id=d91ed101f8354003baf64fb2ad037357)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:33:54.497929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102da71e-8bc4-4498-9876-2e1b75a1144a None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:33:54.499516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 26/55] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:33:54 2022] GET /identity/v3/users/d91ed101f8354003baf64fb2ad037357 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:33:54.504962 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075be853-ea5c-4357-bd69-4c16473d25b6 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:33:54.606801 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075be853-ea5c-4357-bd69-4c16473d25b6 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:33:54.642884 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e70bd0-0e20-4376-be06-5674ae04a8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:33:54.643525 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e70bd0-0e20-4376-be06-5674ae04a8c7 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:54.643810 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e70bd0-0e20-4376-be06-5674ae04a8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:54.644090 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e70bd0-0e20-4376-be06-5674ae04a8c7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:54.645221 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e70bd0-0e20-4376-be06-5674ae04a8c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:33:54.646707 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e70bd0-0e20-4376-be06-5674ae04a8c7 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:33:54.648297 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 30/56] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:33:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:33:54.654567 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102da71e-8bc4-4498-9876-2e1b75a1144a None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:33:54.674146 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:33:54.753250 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102da71e-8bc4-4498-9876-2e1b75a1144a None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:33:54.786176 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e08bf-de0d-4762-8bf4-57979a6b666f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:33:54.786785 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e08bf-de0d-4762-8bf4-57979a6b666f None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:54.787067 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e08bf-de0d-4762-8bf4-57979a6b666f None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:54.787342 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e08bf-de0d-4762-8bf4-57979a6b666f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:54.788063 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e08bf-de0d-4762-8bf4-57979a6b666f None admin] RBAC: Adding query filter params (role.id=6641e95fdc1f4c098ebc372bb692d227, scope.domain.id=default, user.id=d91ed101f8354003baf64fb2ad037357) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:33:54.788370 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e08bf-de0d-4762-8bf4-57979a6b666f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:33:54.789910 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e08bf-de0d-4762-8bf4-57979a6b666f None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:33:54.797380 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 27/57] 10.222.0.76 () {56 vars in 1322 bytes} [Wed Aug 10 22:33:54 2022] GET /identity/v3/role_assignments?user.id=d91ed101f8354003baf64fb2ad037357&scope.domain.id=default&role.id=6641e95fdc1f4c098ebc372bb692d227 => generated 530 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:33:56.180912 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad013d31-5f83-442c-8045-488a7911eb00 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:56.181467 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad013d31-5f83-442c-8045-488a7911eb00 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:56.181467 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad013d31-5f83-442c-8045-488a7911eb00 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:56.182821 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 31/58] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:33:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:33:56.188911 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f4ca9-d7fa-4d20-b6b7-889528432b9e None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:56.189289 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f4ca9-d7fa-4d20-b6b7-889528432b9e None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:56.189501 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f4ca9-d7fa-4d20-b6b7-889528432b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:56.208583 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-d80f4ca9-d7fa-4d20-b6b7-889528432b9e None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:33:56.209833 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-d80f4ca9-d7fa-4d20-b6b7-889528432b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:33:56.274940 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 28/59] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:33:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:33:56.300793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad013d31-5f83-442c-8045-488a7911eb00 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:33:56.346497 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:33:56.403591 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad013d31-5f83-442c-8045-488a7911eb00 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:33:56.437304 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f065a-cf1a-4ca9-a1bf-acf5ae12dc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:33:56.437884 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f065a-cf1a-4ca9-a1bf-acf5ae12dc8d None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:56.438169 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f065a-cf1a-4ca9-a1bf-acf5ae12dc8d None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:56.438447 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f065a-cf1a-4ca9-a1bf-acf5ae12dc8d None admin] PATH_INFO: `/v3/domains/default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:56.439589 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f065a-cf1a-4ca9-a1bf-acf5ae12dc8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:33:56.441031 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f065a-cf1a-4ca9-a1bf-acf5ae12dc8d None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:33:56.442638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 32/60] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:33:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:33:56.448216 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f4ca9-d7fa-4d20-b6b7-889528432b9e None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:33:56.462820 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:33:56.548745 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f4ca9-d7fa-4d20-b6b7-889528432b9e None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:33:56.582681 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e441fb0a-9f9d-432c-9f39-bb7bbf742f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:33:56.583294 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441fb0a-9f9d-432c-9f39-bb7bbf742f1d None admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:56.583578 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441fb0a-9f9d-432c-9f39-bb7bbf742f1d None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:56.583856 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441fb0a-9f9d-432c-9f39-bb7bbf742f1d None admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:56.584508 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e441fb0a-9f9d-432c-9f39-bb7bbf742f1d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:33:56.585990 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e441fb0a-9f9d-432c-9f39-bb7bbf742f1d None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:33:56.612799 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 29/61] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 10 22:33:56 2022] POST /identity/v3/users => generated 275 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:33:59.846256 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a135729-6eda-4f6a-b9e9-7b73d3184f1f None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:59.847324 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a135729-6eda-4f6a-b9e9-7b73d3184f1f None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:59.847840 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a135729-6eda-4f6a-b9e9-7b73d3184f1f None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:59.849475 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 33/62] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:33:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:33:59.855052 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84972b-6a95-4439-b513-41971e948db3 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:59.855655 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84972b-6a95-4439-b513-41971e948db3 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:59.855952 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84972b-6a95-4439-b513-41971e948db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:59.876445 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb50300-80a2-41c9-a926-55183e3dc524 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:59.877227 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb50300-80a2-41c9-a926-55183e3dc524 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:59.879855 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb50300-80a2-41c9-a926-55183e3dc524 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:59.881461 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 34/63] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:33:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:33:59.883437 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-db84972b-6a95-4439-b513-41971e948db3 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:33:59.884887 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-db84972b-6a95-4439-b513-41971e948db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:33:59.890996 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c96a55-651e-43aa-b3eb-3516023646eb None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:33:59.891577 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c96a55-651e-43aa-b3eb-3516023646eb None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:33:59.892057 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c96a55-651e-43aa-b3eb-3516023646eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:33:59.922597 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-18c96a55-651e-43aa-b3eb-3516023646eb None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:33:59.925742 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-18c96a55-651e-43aa-b3eb-3516023646eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:00.017743 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:00.027149 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 30/64] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:33:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:00.051965 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 35/65] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:33:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 10 22:34:00.052162 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84972b-6a95-4439-b513-41971e948db3 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:00.077268 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c96a55-651e-43aa-b3eb-3516023646eb None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:00.131417 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:00.195477 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c96a55-651e-43aa-b3eb-3516023646eb None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:00.198239 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84972b-6a95-4439-b513-41971e948db3 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:00.245615 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa6749-73e9-4c6f-a6de-a8c0b552230b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:00.246030 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa6749-73e9-4c6f-a6de-a8c0b552230b None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:00.246306 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa6749-73e9-4c6f-a6de-a8c0b552230b None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:00.246574 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa6749-73e9-4c6f-a6de-a8c0b552230b None admin] PATH_INFO: `/v3/roles/member` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:00.249647 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29295e41-2565-4b8a-a79b-16586c71de1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:00.250296 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29295e41-2565-4b8a-a79b-16586c71de1d None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:00.250577 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29295e41-2565-4b8a-a79b-16586c71de1d None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:00.250851 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29295e41-2565-4b8a-a79b-16586c71de1d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:00.262217 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fa6749-73e9-4c6f-a6de-a8c0b552230b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:00.262379 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29295e41-2565-4b8a-a79b-16586c71de1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:00.267621 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-29295e41-2565-4b8a-a79b-16586c71de1d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:34:00.267621 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:00.267621 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:00.267621 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:00.267621 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:00.267621 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:00.267621 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:00.267621 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:00.267621 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:00.267621 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:00.267621 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:00.267621 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:00.267621 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:00.267621 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:00.267621 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:00.267621 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:00.268462 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:00.268462 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:00.268462 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:00.268462 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:00.268462 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:00.268462 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:00.268462 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:00.268462 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:00.268462 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:00.268462 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:00.268462 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:00.268462 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:00.268462 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:00.268462 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:00.268462 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:00.268462 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:00.269333 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:00.269333 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:00.269333 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:00.269333 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:00.269333 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:00.269333 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:00.269333 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:00.269333 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:00.269333 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:00.269333 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:00.269333 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:34:00.269333 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:00.269333 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 31/66] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:34:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:34:00.270451 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d4d45-6bbc-4058-904a-cdf5f4b88784 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:00.270741 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d4d45-6bbc-4058-904a-cdf5f4b88784 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:00.271018 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d4d45-6bbc-4058-904a-cdf5f4b88784 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:00.272414 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 32/67] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:00.272703 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-d1fa6749-73e9-4c6f-a6de-a8c0b552230b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:00.272703 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:00.272703 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:00.272703 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:00.272703 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:00.272703 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:00.272703 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:00.272703 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:00.272703 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:00.272703 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:00.272703 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:00.272703 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:00.272703 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:00.272703 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:00.272703 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:00.272703 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:00.273874 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:00.273874 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:00.273874 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:00.273874 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:00.273874 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:00.273874 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:00.273874 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:00.273874 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:00.273874 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:00.273874 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:00.273874 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:00.273874 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:00.273874 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:00.273874 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:00.273874 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:00.273874 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:00.274628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:00.274628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:00.274628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:00.274628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:00.274628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:00.274628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:00.274628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:00.274628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:00.274628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:00.274628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:00.274628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:00.274628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:00.277154 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d4d45-6bbc-4058-904a-cdf5f4b88784 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:00.283579 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 36/68] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:00.283579 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc96f98-cd1c-401e-a3c9-e7c08aba398d None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:00.283579 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc96f98-cd1c-401e-a3c9-e7c08aba398d None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:00.283820 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc96f98-cd1c-401e-a3c9-e7c08aba398d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:00.302270 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-3fc96f98-cd1c-401e-a3c9-e7c08aba398d None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:00.303523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-3fc96f98-cd1c-401e-a3c9-e7c08aba398d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:00.353774 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:00.376228 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 37/69] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:00.377085 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc96f98-cd1c-401e-a3c9-e7c08aba398d None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:00.409256 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d4d45-6bbc-4058-904a-cdf5f4b88784 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:00.413770 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:00.453066 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f5582-0e24-46b9-9025-f72e84a6a637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:00.453745 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f5582-0e24-46b9-9025-f72e84a6a637 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:00.454030 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f5582-0e24-46b9-9025-f72e84a6a637 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:00.454306 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f5582-0e24-46b9-9025-f72e84a6a637 None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:00.454976 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4f5582-0e24-46b9-9025-f72e84a6a637 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:00.455280 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4f5582-0e24-46b9-9025-f72e84a6a637 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:00.456683 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4f5582-0e24-46b9-9025-f72e84a6a637 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:00.465517 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 33/70] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:00.471429 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40bc61-4d00-49a2-afe9-abb7bdb2e0b6 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:00.471716 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40bc61-4d00-49a2-afe9-abb7bdb2e0b6 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:00.471993 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40bc61-4d00-49a2-afe9-abb7bdb2e0b6 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:00.473982 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 34/71] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:00.477868 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f40bc61-4d00-49a2-afe9-abb7bdb2e0b6 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:00.492115 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc96f98-cd1c-401e-a3c9-e7c08aba398d None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:00.526515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28acc4f-29be-4faf-9f1c-0bc30be5fc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:00.527129 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28acc4f-29be-4faf-9f1c-0bc30be5fc60 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:00.527410 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28acc4f-29be-4faf-9f1c-0bc30be5fc60 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:00.527685 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28acc4f-29be-4faf-9f1c-0bc30be5fc60 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:00.528367 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28acc4f-29be-4faf-9f1c-0bc30be5fc60 None admin] RBAC: Adding query filter params (name=member) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:00.528670 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28acc4f-29be-4faf-9f1c-0bc30be5fc60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:00.530388 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28acc4f-29be-4faf-9f1c-0bc30be5fc60 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:00.538948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 38/72] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 10 22:34:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:00.539754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28acc4f-29be-4faf-9f1c-0bc30be5fc60 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:00.589957 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f40bc61-4d00-49a2-afe9-abb7bdb2e0b6 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:00.612053 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:00.630382 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94486d39-9bfe-4e36-8da5-51ac50e4f553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:00.630990 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94486d39-9bfe-4e36-8da5-51ac50e4f553 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:00.631279 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94486d39-9bfe-4e36-8da5-51ac50e4f553 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:00.631561 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94486d39-9bfe-4e36-8da5-51ac50e4f553 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:00.643646 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94486d39-9bfe-4e36-8da5-51ac50e4f553 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:00.646098 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:00.648568 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-94486d39-9bfe-4e36-8da5-51ac50e4f553 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:34:00.648568 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:00.648568 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:00.648568 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:00.648568 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:00.648568 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:00.648568 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:00.648568 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:00.648568 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:00.648568 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:00.648568 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:00.648568 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:00.648568 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:00.648568 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:00.648568 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:00.648568 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:00.649626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:00.649626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:00.649626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:00.649626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:00.649626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:00.649626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:00.649626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:00.649626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:00.649626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:00.649626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:00.649626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:00.649626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:00.649626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:00.649626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:00.649626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:00.649626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:00.656463 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:00.656463 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:00.656463 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:00.656463 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:00.656463 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:00.656463 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:00.656463 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:00.656463 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:00.656463 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:00.656463 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:00.656463 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:34:00.656463 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:00.656463 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 35/73] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 10 22:34:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:00.656463 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641781b-fd6b-429b-8781-92e5e42ee3f8 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:00.657221 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641781b-fd6b-429b-8781-92e5e42ee3f8 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:00.657221 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641781b-fd6b-429b-8781-92e5e42ee3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:00.661523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28acc4f-29be-4faf-9f1c-0bc30be5fc60 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:00.672995 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-5641781b-fd6b-429b-8781-92e5e42ee3f8 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:00.674253 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-5641781b-fd6b-429b-8781-92e5e42ee3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:00.699065 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e21f0-e4d9-4340-82d7-ae906d9e8ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:00.699915 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e21f0-e4d9-4340-82d7-ae906d9e8ba3 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:00.700301 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e21f0-e4d9-4340-82d7-ae906d9e8ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:00.700579 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e21f0-e4d9-4340-82d7-ae906d9e8ba3 None admin] PATH_INFO: `/v3/users/d91ed101f8354003baf64fb2ad037357` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:00.702455 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2e21f0-e4d9-4340-82d7-ae906d9e8ba3 None admin] RBAC: Authorizing `identity:get_user(user_id=d91ed101f8354003baf64fb2ad037357)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:00.703982 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2e21f0-e4d9-4340-82d7-ae906d9e8ba3 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:00.705660 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 39/74] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:34:00 2022] GET /identity/v3/users/d91ed101f8354003baf64fb2ad037357 => generated 247 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:00.710467 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759b11f-3f57-47b6-a565-98fbec10b79d None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:00.710785 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759b11f-3f57-47b6-a565-98fbec10b79d None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:00.711069 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759b11f-3f57-47b6-a565-98fbec10b79d None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:00.712376 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 40/75] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:00.714733 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8759b11f-3f57-47b6-a565-98fbec10b79d None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:00.754534 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 36/76] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:00.755498 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641781b-fd6b-429b-8781-92e5e42ee3f8 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:00.838074 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8759b11f-3f57-47b6-a565-98fbec10b79d None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:00.864028 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641781b-fd6b-429b-8781-92e5e42ee3f8 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:00.873021 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d1154-424a-4450-a100-33441dc2ad81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:00.873703 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d1154-424a-4450-a100-33441dc2ad81 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:00.873990 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d1154-424a-4450-a100-33441dc2ad81 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:00.874269 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d1154-424a-4450-a100-33441dc2ad81 None admin] PATH_INFO: `/v3/users/6e08ec69331f4a57817e3de09ab9221e` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:00.874528 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:00.890354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d1154-424a-4450-a100-33441dc2ad81 None admin] RBAC: Authorizing `identity:get_user(user_id=6e08ec69331f4a57817e3de09ab9221e)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:00.891818 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d1154-424a-4450-a100-33441dc2ad81 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:00.893519 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 41/77] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:34:00 2022] GET /identity/v3/users/6e08ec69331f4a57817e3de09ab9221e => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:00.894302 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d1154-424a-4450-a100-33441dc2ad81 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:00.898945 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc127180-134d-411f-a9eb-e261bab80e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:00.899563 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc127180-134d-411f-a9eb-e261bab80e36 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:00.899842 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc127180-134d-411f-a9eb-e261bab80e36 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:00.900116 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc127180-134d-411f-a9eb-e261bab80e36 None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:00.900815 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc127180-134d-411f-a9eb-e261bab80e36 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:00.901120 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc127180-134d-411f-a9eb-e261bab80e36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:00.902578 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc127180-134d-411f-a9eb-e261bab80e36 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:00.913869 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 37/78] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:34:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:00.914596 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:00.921202 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c3118-8570-45a8-855f-4ba7d28241ad None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:00.921517 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c3118-8570-45a8-855f-4ba7d28241ad None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:00.921799 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c3118-8570-45a8-855f-4ba7d28241ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:00.944548 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-497c3118-8570-45a8-855f-4ba7d28241ad None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:00.945823 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-497c3118-8570-45a8-855f-4ba7d28241ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:01.008477 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d1154-424a-4450-a100-33441dc2ad81 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:01.020620 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 38/79] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:01.021533 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c3118-8570-45a8-855f-4ba7d28241ad None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:01.049092 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800fdfd-c80b-4a71-9057-976cccb12389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:01.049793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800fdfd-c80b-4a71-9057-976cccb12389 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:01.050076 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800fdfd-c80b-4a71-9057-976cccb12389 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:01.050380 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800fdfd-c80b-4a71-9057-976cccb12389 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:01.120964 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e800fdfd-c80b-4a71-9057-976cccb12389 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:01.122500 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e800fdfd-c80b-4a71-9057-976cccb12389 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:01.138259 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c3118-8570-45a8-855f-4ba7d28241ad None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:01.140552 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:01.141209 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:01.176425 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a72bf-ea29-4d49-b0d8-1cc65363483f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:01.176982 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a72bf-ea29-4d49-b0d8-1cc65363483f None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:01.177269 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a72bf-ea29-4d49-b0d8-1cc65363483f None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:01.177577 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a72bf-ea29-4d49-b0d8-1cc65363483f None admin] PATH_INFO: `/v3/roles/member` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:01.187896 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a72bf-ea29-4d49-b0d8-1cc65363483f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:01.192519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-453a72bf-ea29-4d49-b0d8-1cc65363483f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:01.192519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:01.192519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:01.192519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:01.192519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:01.192519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:01.192519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:01.192519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:01.192519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:01.192519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:01.192519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:01.192519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:01.192519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:01.192519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:01.192519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:01.192519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:01.193551 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:01.193551 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:01.193551 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:01.193551 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:01.193551 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:01.193551 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:01.193551 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:01.193551 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:01.193551 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:01.193551 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:01.193551 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:01.193551 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:01.193551 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:01.193551 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:01.193551 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:01.193551 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:01.199797 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:01.199797 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:01.199797 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:01.199797 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:01.199797 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:01.199797 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:01.199797 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:01.199797 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:01.199797 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:01.199797 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:01.199797 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:01.199797 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:01.199797 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 39/80] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:01.199797 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a72bf-ea29-4d49-b0d8-1cc65363483f None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:01.225923 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 42/81] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:34:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:34:01.226670 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800fdfd-c80b-4a71-9057-976cccb12389 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:01.295800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a72bf-ea29-4d49-b0d8-1cc65363483f None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:01.324425 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800fdfd-c80b-4a71-9057-976cccb12389 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:01.329945 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862dc629-75c7-411c-8cb2-452c69e6321c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:01.330559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862dc629-75c7-411c-8cb2-452c69e6321c None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:01.330841 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862dc629-75c7-411c-8cb2-452c69e6321c None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:01.331111 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862dc629-75c7-411c-8cb2-452c69e6321c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:01.343639 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:01.360574 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:01.362196 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b43ea2d-5077-4957-b230-c240c284cd2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:01.362905 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43ea2d-5077-4957-b230-c240c284cd2f None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:01.363192 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43ea2d-5077-4957-b230-c240c284cd2f None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:01.363481 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43ea2d-5077-4957-b230-c240c284cd2f None admin] PATH_INFO: `/v3/users/6e08ec69331f4a57817e3de09ab9221e` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:01.364606 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b43ea2d-5077-4957-b230-c240c284cd2f None admin] RBAC: Authorizing `identity:get_user(user_id=6e08ec69331f4a57817e3de09ab9221e)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:01.366094 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b43ea2d-5077-4957-b230-c240c284cd2f None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:01.367878 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 43/82] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:34:01 2022] GET /identity/v3/users/6e08ec69331f4a57817e3de09ab9221e => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:01.368446 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b43ea2d-5077-4957-b230-c240c284cd2f None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:01.401632 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862dc629-75c7-411c-8cb2-452c69e6321c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:01.403111 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862dc629-75c7-411c-8cb2-452c69e6321c None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:01.469889 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b43ea2d-5077-4957-b230-c240c284cd2f None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:01.502675 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 40/83] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:34:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:34:01.503397 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862dc629-75c7-411c-8cb2-452c69e6321c None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:01.510549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee6e61-7ec6-434e-a2e0-73a3ac22d2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:01.510549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee6e61-7ec6-434e-a2e0-73a3ac22d2e7 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:01.510549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee6e61-7ec6-434e-a2e0-73a3ac22d2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:01.510549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee6e61-7ec6-434e-a2e0-73a3ac22d2e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:01.510899 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee6e61-7ec6-434e-a2e0-73a3ac22d2e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:01.511943 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee6e61-7ec6-434e-a2e0-73a3ac22d2e7 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:01.513587 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 44/84] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:34:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:01.515460 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee6e61-7ec6-434e-a2e0-73a3ac22d2e7 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:01.566244 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:01.572567 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:01.606301 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862dc629-75c7-411c-8cb2-452c69e6321c None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:01.618087 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee6e61-7ec6-434e-a2e0-73a3ac22d2e7 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:01.639949 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a068e-16e8-4037-9394-f68a70689a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:01.640558 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a068e-16e8-4037-9394-f68a70689a6d None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:01.640840 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a068e-16e8-4037-9394-f68a70689a6d None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:01.641117 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a068e-16e8-4037-9394-f68a70689a6d None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:01.641804 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a068e-16e8-4037-9394-f68a70689a6d None admin] RBAC: Adding query filter params (name=member) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:01.642104 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a068e-16e8-4037-9394-f68a70689a6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:01.643496 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a068e-16e8-4037-9394-f68a70689a6d None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:01.651346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 41/85] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 10 22:34:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:01.652155 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a068e-16e8-4037-9394-f68a70689a6d None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:01.655488 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a9846-ec30-4217-9eeb-475f86d7777f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:01.656197 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a9846-ec30-4217-9eeb-475f86d7777f None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:01.656477 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a9846-ec30-4217-9eeb-475f86d7777f None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:01.656749 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a9846-ec30-4217-9eeb-475f86d7777f None admin] PATH_INFO: `/v3/projects/2694f62f168849c0a5cd44bffc841cef` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:01.657982 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a9846-ec30-4217-9eeb-475f86d7777f None admin] RBAC: Authorizing `identity:get_project(project_id=2694f62f168849c0a5cd44bffc841cef)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:01.659451 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a9846-ec30-4217-9eeb-475f86d7777f None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:01.665734 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 45/86] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:01 2022] GET /identity/v3/projects/2694f62f168849c0a5cd44bffc841cef => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:01.665734 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a9846-ec30-4217-9eeb-475f86d7777f None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:01.755950 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a068e-16e8-4037-9394-f68a70689a6d None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:01.762307 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a9846-ec30-4217-9eeb-475f86d7777f None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:01.779195 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:01.786898 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:01.790358 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f112f7e-5c74-46a7-a47b-01cbcb33b8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:01.790969 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f112f7e-5c74-46a7-a47b-01cbcb33b8b5 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:01.791254 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f112f7e-5c74-46a7-a47b-01cbcb33b8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:01.791523 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f112f7e-5c74-46a7-a47b-01cbcb33b8b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:01.799933 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3185b930-5fd6-44cb-b2d6-668c25d875c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:01.800966 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185b930-5fd6-44cb-b2d6-668c25d875c4 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:01.800966 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185b930-5fd6-44cb-b2d6-668c25d875c4 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:01.801142 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185b930-5fd6-44cb-b2d6-668c25d875c4 None admin] PATH_INFO: `/v3/projects/2694f62f168849c0a5cd44bffc841cef` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:01.802286 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3185b930-5fd6-44cb-b2d6-668c25d875c4 None admin] RBAC: Authorizing `identity:get_project(project_id=2694f62f168849c0a5cd44bffc841cef)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:01.803761 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3185b930-5fd6-44cb-b2d6-668c25d875c4 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:01.805427 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 46/87] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:01 2022] GET /identity/v3/projects/2694f62f168849c0a5cd44bffc841cef => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:01.805427 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3185b930-5fd6-44cb-b2d6-668c25d875c4 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:01.860469 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f112f7e-5c74-46a7-a47b-01cbcb33b8b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:01.862010 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f112f7e-5c74-46a7-a47b-01cbcb33b8b5 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:01.910344 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3185b930-5fd6-44cb-b2d6-668c25d875c4 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:01.945233 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ab4de-cd6b-45bd-99b5-62f7384aee31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:01.945907 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ab4de-cd6b-45bd-99b5-62f7384aee31 None admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:01.946197 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ab4de-cd6b-45bd-99b5-62f7384aee31 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:01.946473 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ab4de-cd6b-45bd-99b5-62f7384aee31 None admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:01.947119 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4ab4de-cd6b-45bd-99b5-62f7384aee31 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:01.948561 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4ab4de-cd6b-45bd-99b5-62f7384aee31 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:01.960511 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 42/88] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:34:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:34:01.961389 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f112f7e-5c74-46a7-a47b-01cbcb33b8b5 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:01.977987 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 47/89] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 10 22:34:01 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:34:01.978820 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ab4de-cd6b-45bd-99b5-62f7384aee31 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:01.988212 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:02.042650 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:02.079457 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f112f7e-5c74-46a7-a47b-01cbcb33b8b5 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:02.108133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ab4de-cd6b-45bd-99b5-62f7384aee31 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:02.119569 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a19e42-cf34-44f4-b150-786798804508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:02.120553 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a19e42-cf34-44f4-b150-786798804508 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:02.121004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a19e42-cf34-44f4-b150-786798804508 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:02.121306 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a19e42-cf34-44f4-b150-786798804508 None admin] PATH_INFO: `/v3/users/6e08ec69331f4a57817e3de09ab9221e` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:02.122531 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a19e42-cf34-44f4-b150-786798804508 None admin] RBAC: Authorizing `identity:get_user(user_id=6e08ec69331f4a57817e3de09ab9221e)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:02.124826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a19e42-cf34-44f4-b150-786798804508 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:02.131680 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 43/90] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:34:01 2022] GET /identity/v3/users/6e08ec69331f4a57817e3de09ab9221e => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:02.131680 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a19e42-cf34-44f4-b150-786798804508 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:02.155995 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3488ec-b65c-4fda-b808-46a53b854e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:02.156731 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3488ec-b65c-4fda-b808-46a53b854e7d None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:02.157014 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3488ec-b65c-4fda-b808-46a53b854e7d None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:02.157299 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3488ec-b65c-4fda-b808-46a53b854e7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:02.158058 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3488ec-b65c-4fda-b808-46a53b854e7d None admin] RBAC: Adding query filter params (role.id=6641e95fdc1f4c098ebc372bb692d227, scope.project.id=2694f62f168849c0a5cd44bffc841cef, user.id=d91ed101f8354003baf64fb2ad037357) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:02.158852 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3488ec-b65c-4fda-b808-46a53b854e7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:02.159831 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3488ec-b65c-4fda-b808-46a53b854e7d None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:02.166683 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 48/91] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:34:01 2022] GET /identity/v3/role_assignments?user.id=d91ed101f8354003baf64fb2ad037357&scope.project.id=2694f62f168849c0a5cd44bffc841cef&role.id=6641e95fdc1f4c098ebc372bb692d227 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:02.167590 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3488ec-b65c-4fda-b808-46a53b854e7d None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:02.259776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:02.269531 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a19e42-cf34-44f4-b150-786798804508 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:02.304587 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3488ec-b65c-4fda-b808-46a53b854e7d None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:02.308623 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ece25-b50a-419b-acc8-3c2c06746af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:02.309261 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ece25-b50a-419b-acc8-3c2c06746af2 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:02.309573 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ece25-b50a-419b-acc8-3c2c06746af2 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:02.309847 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ece25-b50a-419b-acc8-3c2c06746af2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:02.310590 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7ece25-b50a-419b-acc8-3c2c06746af2 None admin] RBAC: Adding query filter params (role.id=1085473c6e534f87812b05c7014f77c2, scope.project.id=2694f62f168849c0a5cd44bffc841cef, user.id=6e08ec69331f4a57817e3de09ab9221e) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:02.310893 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7ece25-b50a-419b-acc8-3c2c06746af2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:02.312920 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7ece25-b50a-419b-acc8-3c2c06746af2 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:02.318255 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:02.320636 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 44/92] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:34:02 2022] GET /identity/v3/role_assignments?user.id=6e08ec69331f4a57817e3de09ab9221e&scope.project.id=2694f62f168849c0a5cd44bffc841cef&role.id=1085473c6e534f87812b05c7014f77c2 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:02.321908 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ece25-b50a-419b-acc8-3c2c06746af2 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:02.356488 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e5bd02-2925-4e27-bbed-8e80f3feed5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:02.357855 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5bd02-2925-4e27-bbed-8e80f3feed5c None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:02.359024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5bd02-2925-4e27-bbed-8e80f3feed5c None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:02.359544 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5bd02-2925-4e27-bbed-8e80f3feed5c None admin] PATH_INFO: `/v3/projects/2694f62f168849c0a5cd44bffc841cef` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:02.361687 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e5bd02-2925-4e27-bbed-8e80f3feed5c None admin] RBAC: Authorizing `identity:get_project(project_id=2694f62f168849c0a5cd44bffc841cef)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:02.364299 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e5bd02-2925-4e27-bbed-8e80f3feed5c None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:02.366768 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 49/93] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:02 2022] GET /identity/v3/projects/2694f62f168849c0a5cd44bffc841cef => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:02.373420 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e5bd02-2925-4e27-bbed-8e80f3feed5c None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:02.464687 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ece25-b50a-419b-acc8-3c2c06746af2 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:02.510881 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabeea74-9ef5-4167-9fcc-ce075f8233d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:02.512001 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabeea74-9ef5-4167-9fcc-ce075f8233d0 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:02.512580 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabeea74-9ef5-4167-9fcc-ce075f8233d0 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:02.513066 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabeea74-9ef5-4167-9fcc-ce075f8233d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:02.516937 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e5bd02-2925-4e27-bbed-8e80f3feed5c None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:02.553520 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:02.558119 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6625d-c42c-46d1-b57b-6a1b5157b721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:02.558884 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6625d-c42c-46d1-b57b-6a1b5157b721 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:02.559269 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6625d-c42c-46d1-b57b-6a1b5157b721 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:02.559721 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6625d-c42c-46d1-b57b-6a1b5157b721 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:02.560493 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c6625d-c42c-46d1-b57b-6a1b5157b721 None admin] RBAC: Adding query filter params (role.id=e22e4d2b3d06480bbcfa4d2ee3387c90, scope.project.id=2694f62f168849c0a5cd44bffc841cef, user.id=6e08ec69331f4a57817e3de09ab9221e) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:02.560912 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c6625d-c42c-46d1-b57b-6a1b5157b721 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:02.562706 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c6625d-c42c-46d1-b57b-6a1b5157b721 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:02.569552 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 50/94] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:34:02 2022] GET /identity/v3/role_assignments?user.id=6e08ec69331f4a57817e3de09ab9221e&scope.project.id=2694f62f168849c0a5cd44bffc841cef&role.id=e22e4d2b3d06480bbcfa4d2ee3387c90 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:02.617596 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabeea74-9ef5-4167-9fcc-ce075f8233d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:02.619115 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabeea74-9ef5-4167-9fcc-ce075f8233d0 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:02.749460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 45/95] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:34:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:34:02.760656 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6625d-c42c-46d1-b57b-6a1b5157b721 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:02.823687 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:02.897964 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6625d-c42c-46d1-b57b-6a1b5157b721 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:02.943561 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5720294-f19c-47f9-a127-294f37edab94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:02.944328 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5720294-f19c-47f9-a127-294f37edab94 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:02.944551 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5720294-f19c-47f9-a127-294f37edab94 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:02.944879 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5720294-f19c-47f9-a127-294f37edab94 None admin] PATH_INFO: `/v3/projects/88da906d760243afb9b93d85aaabfa7e` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:02.946136 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5720294-f19c-47f9-a127-294f37edab94 None admin] RBAC: Authorizing `identity:get_project(project_id=88da906d760243afb9b93d85aaabfa7e)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:02.947607 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5720294-f19c-47f9-a127-294f37edab94 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:02.949585 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 51/96] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:02 2022] GET /identity/v3/projects/88da906d760243afb9b93d85aaabfa7e => generated 312 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:02.955783 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabeea74-9ef5-4167-9fcc-ce075f8233d0 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:03.055268 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:03.116422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabeea74-9ef5-4167-9fcc-ce075f8233d0 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:03.170177 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b13eb-40b6-411a-a574-8660b7c50ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:03.170803 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b13eb-40b6-411a-a574-8660b7c50ec1 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:03.171779 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b13eb-40b6-411a-a574-8660b7c50ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:03.172067 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b13eb-40b6-411a-a574-8660b7c50ec1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:03.172797 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b13eb-40b6-411a-a574-8660b7c50ec1 None admin] RBAC: Adding query filter params (role.id=1085473c6e534f87812b05c7014f77c2, scope.project.id=88da906d760243afb9b93d85aaabfa7e, user.id=6e08ec69331f4a57817e3de09ab9221e) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:03.173100 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b13eb-40b6-411a-a574-8660b7c50ec1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:03.174586 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b13eb-40b6-411a-a574-8660b7c50ec1 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:03.180531 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 46/97] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:34:02 2022] GET /identity/v3/role_assignments?user.id=6e08ec69331f4a57817e3de09ab9221e&scope.project.id=88da906d760243afb9b93d85aaabfa7e&role.id=1085473c6e534f87812b05c7014f77c2 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:04.778995 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce040189-c9bd-4273-a90d-028dfc6a5f59 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:04.779670 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce040189-c9bd-4273-a90d-028dfc6a5f59 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:04.779670 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce040189-c9bd-4273-a90d-028dfc6a5f59 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:04.780843 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 52/98] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:04.786630 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8e92a-208f-4f25-bd9c-2ae0db7a299b None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:04.787163 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8e92a-208f-4f25-bd9c-2ae0db7a299b None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:04.787163 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8e92a-208f-4f25-bd9c-2ae0db7a299b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:04.814481 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-f5c8e92a-208f-4f25-bd9c-2ae0db7a299b None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:04.815739 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-f5c8e92a-208f-4f25-bd9c-2ae0db7a299b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:04.904003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 47/99] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:04.938723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce040189-c9bd-4273-a90d-028dfc6a5f59 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:05.095077 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce040189-c9bd-4273-a90d-028dfc6a5f59 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:05.110760 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:05.147130 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dd9aff-e469-4271-96fb-1e3feafe6683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:05.147794 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd9aff-e469-4271-96fb-1e3feafe6683 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:05.148078 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd9aff-e469-4271-96fb-1e3feafe6683 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:05.148356 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd9aff-e469-4271-96fb-1e3feafe6683 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:05.166200 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dd9aff-e469-4271-96fb-1e3feafe6683 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:05.170986 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-05dd9aff-e469-4271-96fb-1e3feafe6683 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:34:05.170986 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:05.170986 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:05.170986 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:05.170986 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:05.170986 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:05.170986 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:05.170986 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:05.170986 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:05.170986 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:05.170986 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:05.170986 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:05.170986 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:05.170986 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:05.170986 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:05.170986 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:05.172170 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:05.172170 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:05.172170 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:05.172170 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:05.172170 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:05.172170 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:05.172170 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:05.172170 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:05.172170 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:05.172170 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:05.172170 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:05.172170 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:05.172170 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:05.172170 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:05.172170 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:05.172170 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:05.173153 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:05.173153 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:05.173153 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:05.173153 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:05.173153 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:05.173153 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:05.173153 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:05.173153 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:05.173153 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:05.173153 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:05.173153 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:34:05.173153 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:05.173153 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 53/100] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:05.180505 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c8e92a-208f-4f25-bd9c-2ae0db7a299b None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:05.261931 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:05.312272 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec26e9-149e-41ba-a856-c353c6e01e96 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:05.312624 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec26e9-149e-41ba-a856-c353c6e01e96 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:05.312899 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec26e9-149e-41ba-a856-c353c6e01e96 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:05.314253 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 54/101] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:34:05.321010 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4aadf5-1357-483f-bbe9-51d8aee2c061 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:05.321313 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4aadf5-1357-483f-bbe9-51d8aee2c061 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:05.321613 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4aadf5-1357-483f-bbe9-51d8aee2c061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:05.342538 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c8e92a-208f-4f25-bd9c-2ae0db7a299b None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:05.357642 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-ac4aadf5-1357-483f-bbe9-51d8aee2c061 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:05.358907 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-ac4aadf5-1357-483f-bbe9-51d8aee2c061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:05.409042 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aded79e6-153d-4925-8337-17aaeced76c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:05.409725 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded79e6-153d-4925-8337-17aaeced76c9 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:05.410014 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded79e6-153d-4925-8337-17aaeced76c9 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:05.410291 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded79e6-153d-4925-8337-17aaeced76c9 None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:05.410971 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aded79e6-153d-4925-8337-17aaeced76c9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:05.411284 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aded79e6-153d-4925-8337-17aaeced76c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:05.412680 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aded79e6-153d-4925-8337-17aaeced76c9 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:05.436595 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 48/102] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 10 22:34:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:05.439543 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df7c19-11a4-4567-9310-6ff8b4e8db44 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:05.448226 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df7c19-11a4-4567-9310-6ff8b4e8db44 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:05.448873 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df7c19-11a4-4567-9310-6ff8b4e8db44 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:05.451147 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 49/103] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:05 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:05.452156 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df7c19-11a4-4567-9310-6ff8b4e8db44 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:05.486588 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 55/104] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:34:05.488454 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c72e3-3d01-4079-b892-532fbf3eb56a None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:05.489308 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c72e3-3d01-4079-b892-532fbf3eb56a None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:05.490061 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c72e3-3d01-4079-b892-532fbf3eb56a None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:05.491837 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 56/105] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:05.493064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb0e5c-9c3c-4bfd-a8e2-7f0c2485876d None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:05.493570 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb0e5c-9c3c-4bfd-a8e2-7f0c2485876d None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:05.494050 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb0e5c-9c3c-4bfd-a8e2-7f0c2485876d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:05.525886 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-23cb0e5c-9c3c-4bfd-a8e2-7f0c2485876d None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:05.527723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-23cb0e5c-9c3c-4bfd-a8e2-7f0c2485876d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:05.591275 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df7c19-11a4-4567-9310-6ff8b4e8db44 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:05.608654 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 57/106] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:05.610508 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:05.619384 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43df02-50af-45cd-a18d-cc5506fffcd5 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:05.619785 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43df02-50af-45cd-a18d-cc5506fffcd5 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:05.620153 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43df02-50af-45cd-a18d-cc5506fffcd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:05.651795 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250feab7-3033-4b3a-863d-3f341a5f2334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:05.652749 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250feab7-3033-4b3a-863d-3f341a5f2334 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:05.653321 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250feab7-3033-4b3a-863d-3f341a5f2334 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:05.653693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-1a43df02-50af-45cd-a18d-cc5506fffcd5 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:05.653994 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250feab7-3033-4b3a-863d-3f341a5f2334 None admin] PATH_INFO: `/v3/users/9ddd8e696aa24e88917377eb6e3199da` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:05.655340 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-1a43df02-50af-45cd-a18d-cc5506fffcd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:05.676224 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250feab7-3033-4b3a-863d-3f341a5f2334 None admin] RBAC: Authorizing `identity:get_user(user_id=9ddd8e696aa24e88917377eb6e3199da)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:05.678528 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250feab7-3033-4b3a-863d-3f341a5f2334 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:05.680902 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 50/107] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:34:05 2022] GET /identity/v3/users/9ddd8e696aa24e88917377eb6e3199da => generated 289 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:05.681582 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:05.682577 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250feab7-3033-4b3a-863d-3f341a5f2334 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:05.730919 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 58/108] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:05.737303 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43df02-50af-45cd-a18d-cc5506fffcd5 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:05.822823 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250feab7-3033-4b3a-863d-3f341a5f2334 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:05.864434 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43df02-50af-45cd-a18d-cc5506fffcd5 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:05.871503 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde5f6af-b2dd-4ebf-9a14-b2a365a7ebd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:05.872278 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5f6af-b2dd-4ebf-9a14-b2a365a7ebd4 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:05.872669 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5f6af-b2dd-4ebf-9a14-b2a365a7ebd4 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:05.873051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5f6af-b2dd-4ebf-9a14-b2a365a7ebd4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:05.888987 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde5f6af-b2dd-4ebf-9a14-b2a365a7ebd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:05.896056 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-bde5f6af-b2dd-4ebf-9a14-b2a365a7ebd4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:34:05.896056 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:05.896056 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:05.896056 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:05.896056 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:05.896056 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:05.896056 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:05.896056 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:05.896056 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:05.896056 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:05.896056 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:05.896056 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:05.896056 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:05.896056 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:05.896056 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:05.896056 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:05.896925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:05.896925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:05.896925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:05.896925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:05.896925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:05.896925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:05.896925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:05.896925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:05.896925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:05.896925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:05.896925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:05.896925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:05.896925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:05.896925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:05.896925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:05.896925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:05.897747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:05.897747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:05.897747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:05.897747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:05.897747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:05.897747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:05.897747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:05.897747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:05.897747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:05.897747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:05.897747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:34:05.897747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:05.897747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 51/109] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:34:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:05.898379 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde5f6af-b2dd-4ebf-9a14-b2a365a7ebd4 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:05.916506 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d13e0f-a409-454e-8e12-0d9f192ea2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:05.917585 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d13e0f-a409-454e-8e12-0d9f192ea2d7 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:05.918047 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d13e0f-a409-454e-8e12-0d9f192ea2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:05.918716 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d13e0f-a409-454e-8e12-0d9f192ea2d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:05.920156 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d13e0f-a409-454e-8e12-0d9f192ea2d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:05.922016 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d13e0f-a409-454e-8e12-0d9f192ea2d7 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:05.924387 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 59/110] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:34:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:05.925447 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d13e0f-a409-454e-8e12-0d9f192ea2d7 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:05.952371 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:05.960140 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:06.045815 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d13e0f-a409-454e-8e12-0d9f192ea2d7 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:06.058097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde5f6af-b2dd-4ebf-9a14-b2a365a7ebd4 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:06.085003 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f74c70c-75cf-44c4-ab9c-1ca7535b7822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:06.086293 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74c70c-75cf-44c4-ab9c-1ca7535b7822 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:06.086809 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74c70c-75cf-44c4-ab9c-1ca7535b7822 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:06.087334 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74c70c-75cf-44c4-ab9c-1ca7535b7822 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:06.099312 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f74c70c-75cf-44c4-ab9c-1ca7535b7822 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:06.104660 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce131d-e6e4-4752-82c4-8159342873e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:06.105305 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce131d-e6e4-4752-82c4-8159342873e0 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:06.105599 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce131d-e6e4-4752-82c4-8159342873e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:06.105899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce131d-e6e4-4752-82c4-8159342873e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:06.106484 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-9f74c70c-75cf-44c4-ab9c-1ca7535b7822 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:06.106484 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:06.106484 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:06.106484 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:06.106484 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:06.106484 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:06.106484 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:06.106484 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:06.106484 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:06.106484 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:06.106484 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:06.106484 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:06.106484 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:06.106484 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:06.106484 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:06.106484 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:06.107753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:06.107753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:06.107753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:06.107753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:06.107753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:06.107753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:06.107753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:06.107753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:06.107753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:06.107753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:06.107753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:06.107753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:06.107753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:06.107753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:06.107753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:06.107753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:06.108709 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:06.108709 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:06.108709 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:06.108709 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:06.108709 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:06.108709 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:06.108709 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:06.108709 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:06.108709 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:06.108709 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:06.108709 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:06.108709 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:06.116217 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 60/111] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:06.116217 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f74c70c-75cf-44c4-ab9c-1ca7535b7822 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:06.180757 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ce131d-e6e4-4752-82c4-8159342873e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:06.182409 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ce131d-e6e4-4752-82c4-8159342873e0 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:06.214577 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:06.217484 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:06.231267 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f74c70c-75cf-44c4-ab9c-1ca7535b7822 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:06.270247 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f6673-a3e5-4033-9cf5-54a8938cc6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:06.270864 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f6673-a3e5-4033-9cf5-54a8938cc6d6 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:06.271151 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f6673-a3e5-4033-9cf5-54a8938cc6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:06.271429 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f6673-a3e5-4033-9cf5-54a8938cc6d6 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:06.272123 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f6673-a3e5-4033-9cf5-54a8938cc6d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:06.272410 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f6673-a3e5-4033-9cf5-54a8938cc6d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:06.273848 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f6673-a3e5-4033-9cf5-54a8938cc6d6 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:06.286838 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 61/112] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:06.286838 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f6673-a3e5-4033-9cf5-54a8938cc6d6 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:06.299448 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 52/113] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:34:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:34:06.300819 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66237753-407a-4808-b0e2-d89a20e60afd None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:06.301185 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66237753-407a-4808-b0e2-d89a20e60afd None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:06.301559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66237753-407a-4808-b0e2-d89a20e60afd None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:06.302889 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 53/114] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:06.304923 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66237753-407a-4808-b0e2-d89a20e60afd None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:06.390605 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f6673-a3e5-4033-9cf5-54a8938cc6d6 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:06.405913 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66237753-407a-4808-b0e2-d89a20e60afd None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:06.426357 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0489ee-0e5a-4fc1-a3ee-893096ac17d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:06.426999 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0489ee-0e5a-4fc1-a3ee-893096ac17d3 None admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:06.427274 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0489ee-0e5a-4fc1-a3ee-893096ac17d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:06.427551 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0489ee-0e5a-4fc1-a3ee-893096ac17d3 None admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:06.428204 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0489ee-0e5a-4fc1-a3ee-893096ac17d3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:06.430086 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0489ee-0e5a-4fc1-a3ee-893096ac17d3 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:06.438360 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:06.445233 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b374154-15d9-44d0-973f-b87d928581b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:06.445881 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b374154-15d9-44d0-973f-b87d928581b8 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:06.446166 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b374154-15d9-44d0-973f-b87d928581b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:06.446792 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b374154-15d9-44d0-973f-b87d928581b8 None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:06.447117 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b374154-15d9-44d0-973f-b87d928581b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:06.447470 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b374154-15d9-44d0-973f-b87d928581b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:06.449069 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b374154-15d9-44d0-973f-b87d928581b8 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:06.459110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 54/115] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 10 22:34:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:06.459268 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 62/116] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 10 22:34:06 2022] POST /identity/v3/projects => generated 302 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:34:06.459669 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b374154-15d9-44d0-973f-b87d928581b8 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:06.466914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4cf20-b023-4f3b-ba2a-316f2eabca94 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:06.467180 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4cf20-b023-4f3b-ba2a-316f2eabca94 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:06.467431 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4cf20-b023-4f3b-ba2a-316f2eabca94 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:06.469594 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 63/117] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:06.470316 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4cf20-b023-4f3b-ba2a-316f2eabca94 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:06.476235 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:06.582834 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b374154-15d9-44d0-973f-b87d928581b8 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:06.595477 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4cf20-b023-4f3b-ba2a-316f2eabca94 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:06.622232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8951bd-f1cd-41a6-9c8a-d2ecf7cb163b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:06.622890 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8951bd-f1cd-41a6-9c8a-d2ecf7cb163b None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:06.623177 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8951bd-f1cd-41a6-9c8a-d2ecf7cb163b None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:06.624067 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8951bd-f1cd-41a6-9c8a-d2ecf7cb163b None admin] PATH_INFO: `/v3/users/d91ed101f8354003baf64fb2ad037357` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:06.624613 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8951bd-f1cd-41a6-9c8a-d2ecf7cb163b None admin] RBAC: Authorizing `identity:get_user(user_id=d91ed101f8354003baf64fb2ad037357)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:06.627088 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8951bd-f1cd-41a6-9c8a-d2ecf7cb163b None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:06.629320 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 55/118] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:34:06 2022] GET /identity/v3/users/d91ed101f8354003baf64fb2ad037357 => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:06.629944 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61732f4-d211-45fb-95bd-cbc5657f2296 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:06.630235 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61732f4-d211-45fb-95bd-cbc5657f2296 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:06.630517 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61732f4-d211-45fb-95bd-cbc5657f2296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:06.637915 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145d5ee-3b97-4d07-8c83-a224dc189349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:06.639129 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145d5ee-3b97-4d07-8c83-a224dc189349 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:06.639129 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145d5ee-3b97-4d07-8c83-a224dc189349 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:06.639294 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145d5ee-3b97-4d07-8c83-a224dc189349 None admin] PATH_INFO: `/v3/projects/2694f62f168849c0a5cd44bffc841cef` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:06.640432 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e145d5ee-3b97-4d07-8c83-a224dc189349 None admin] RBAC: Authorizing `identity:get_project(project_id=2694f62f168849c0a5cd44bffc841cef)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:06.642724 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e145d5ee-3b97-4d07-8c83-a224dc189349 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:06.644590 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 64/119] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:06 2022] GET /identity/v3/projects/2694f62f168849c0a5cd44bffc841cef => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:06.645076 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145d5ee-3b97-4d07-8c83-a224dc189349 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:06.653082 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-f61732f4-d211-45fb-95bd-cbc5657f2296 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:06.654442 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-f61732f4-d211-45fb-95bd-cbc5657f2296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:06.720385 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:06.728410 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:06.732822 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 56/120] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:06.740911 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a59d7-b076-4b69-8420-2dfd758b49ee None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:06.740911 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a59d7-b076-4b69-8420-2dfd758b49ee None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:06.740911 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a59d7-b076-4b69-8420-2dfd758b49ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:06.758481 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-877a59d7-b076-4b69-8420-2dfd758b49ee None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:06.758481 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-877a59d7-b076-4b69-8420-2dfd758b49ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:06.778822 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145d5ee-3b97-4d07-8c83-a224dc189349 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:06.822796 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf3ef7-3deb-4499-bbc1-51c4ca627dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:06.823434 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf3ef7-3deb-4499-bbc1-51c4ca627dae None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:06.823720 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf3ef7-3deb-4499-bbc1-51c4ca627dae None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:06.824031 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf3ef7-3deb-4499-bbc1-51c4ca627dae None admin] PATH_INFO: `/v3/users/6e08ec69331f4a57817e3de09ab9221e` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:06.825151 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf3ef7-3deb-4499-bbc1-51c4ca627dae None admin] RBAC: Authorizing `identity:get_user(user_id=6e08ec69331f4a57817e3de09ab9221e)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:06.826652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf3ef7-3deb-4499-bbc1-51c4ca627dae None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:06.830156 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 57/121] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:06.833678 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 65/122] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:34:06 2022] GET /identity/v3/users/6e08ec69331f4a57817e3de09ab9221e => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:06.834419 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf3ef7-3deb-4499-bbc1-51c4ca627dae None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:06.842115 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a59d7-b076-4b69-8420-2dfd758b49ee None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:06.951928 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a59d7-b076-4b69-8420-2dfd758b49ee None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:06.954551 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf3ef7-3deb-4499-bbc1-51c4ca627dae None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:06.974450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:06.990880 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaef611-2e76-48bf-9100-157dad45e10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:06.991058 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e08b5a-6a12-4ad9-9063-b77accdd4f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:06.991502 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e08b5a-6a12-4ad9-9063-b77accdd4f01 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:06.991602 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaef611-2e76-48bf-9100-157dad45e10b None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:06.991788 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e08b5a-6a12-4ad9-9063-b77accdd4f01 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:06.991893 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaef611-2e76-48bf-9100-157dad45e10b None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:06.992060 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e08b5a-6a12-4ad9-9063-b77accdd4f01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:06.992153 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaef611-2e76-48bf-9100-157dad45e10b None admin] PATH_INFO: `/v3/projects/2694f62f168849c0a5cd44bffc841cef` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:06.992782 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e08b5a-6a12-4ad9-9063-b77accdd4f01 None admin] RBAC: Adding query filter params (role.id=05a9201bdeee4f2ab18c648d5e6cb250, scope.project.id=2694f62f168849c0a5cd44bffc841cef, user.id=9ddd8e696aa24e88917377eb6e3199da) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:06.993086 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e08b5a-6a12-4ad9-9063-b77accdd4f01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:06.993190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaef611-2e76-48bf-9100-157dad45e10b None admin] RBAC: Authorizing `identity:get_project(project_id=2694f62f168849c0a5cd44bffc841cef)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:06.994562 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e08b5a-6a12-4ad9-9063-b77accdd4f01 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:06.994657 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaef611-2e76-48bf-9100-157dad45e10b None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:06.996275 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 58/123] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:06 2022] GET /identity/v3/projects/2694f62f168849c0a5cd44bffc841cef => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:06.997022 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaef611-2e76-48bf-9100-157dad45e10b None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:07.006126 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 66/124] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:34:06 2022] GET /identity/v3/role_assignments?user.id=9ddd8e696aa24e88917377eb6e3199da&scope.project.id=2694f62f168849c0a5cd44bffc841cef&role.id=05a9201bdeee4f2ab18c648d5e6cb250 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:07.010081 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e08b5a-6a12-4ad9-9063-b77accdd4f01 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:07.014654 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:07.129649 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaef611-2e76-48bf-9100-157dad45e10b None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:07.169620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e08b5a-6a12-4ad9-9063-b77accdd4f01 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:07.173384 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c6fcd-33f6-41c4-8364-d1dfe02415e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:07.174445 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c6fcd-33f6-41c4-8364-d1dfe02415e1 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:07.174445 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c6fcd-33f6-41c4-8364-d1dfe02415e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:07.174445 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c6fcd-33f6-41c4-8364-d1dfe02415e1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:07.194330 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c6fcd-33f6-41c4-8364-d1dfe02415e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:07.201884 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-af2c6fcd-33f6-41c4-8364-d1dfe02415e1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:34:07.201884 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:07.201884 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:07.201884 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:07.201884 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:07.201884 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:07.201884 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:07.201884 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:07.201884 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:07.201884 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:07.201884 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:07.201884 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:07.201884 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:07.201884 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:07.201884 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:07.201884 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:07.203076 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:07.203076 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:07.203076 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:07.203076 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:07.203076 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:07.203076 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:07.203076 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:07.203076 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:07.203076 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:07.203076 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:07.203076 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:07.203076 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:07.203076 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:07.203076 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:07.203076 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:07.203076 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:07.204051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:07.204051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:07.204051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:07.204051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:07.204051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:07.204051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:07.204051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:07.204051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:07.204051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:07.204051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:07.204051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:34:07.204051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:07.206837 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 59/125] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 10 22:34:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:07.207969 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c6fcd-33f6-41c4-8364-d1dfe02415e1 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:07.219978 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643977c0-3f2c-4f2b-a804-6088fef26bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:07.220739 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643977c0-3f2c-4f2b-a804-6088fef26bff None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:07.221127 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643977c0-3f2c-4f2b-a804-6088fef26bff None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:07.221523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643977c0-3f2c-4f2b-a804-6088fef26bff None admin] PATH_INFO: `/v3/projects/2694f62f168849c0a5cd44bffc841cef` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:07.222921 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643977c0-3f2c-4f2b-a804-6088fef26bff None admin] RBAC: Authorizing `identity:get_project(project_id=2694f62f168849c0a5cd44bffc841cef)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:07.224506 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643977c0-3f2c-4f2b-a804-6088fef26bff None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:07.226454 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 67/126] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:07 2022] GET /identity/v3/projects/2694f62f168849c0a5cd44bffc841cef => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:07.227424 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643977c0-3f2c-4f2b-a804-6088fef26bff None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:07.277654 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:07.300331 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:07.335490 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643977c0-3f2c-4f2b-a804-6088fef26bff None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:07.337926 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c6fcd-33f6-41c4-8364-d1dfe02415e1 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:07.372921 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1952fd96-4046-4cb2-8f44-3990de2d9be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:07.373774 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952fd96-4046-4cb2-8f44-3990de2d9be6 None admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:07.374155 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952fd96-4046-4cb2-8f44-3990de2d9be6 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:07.374740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952fd96-4046-4cb2-8f44-3990de2d9be6 None admin] PATH_INFO: `/v3/projects/2694f62f168849c0a5cd44bffc841cef/users/d91ed101f8354003baf64fb2ad037357/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:07.377350 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1952fd96-4046-4cb2-8f44-3990de2d9be6 None admin] RBAC: Authorizing `identity:create_grant(project_id=2694f62f168849c0a5cd44bffc841cef, user_id=d91ed101f8354003baf64fb2ad037357, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:07.378922 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1952fd96-4046-4cb2-8f44-3990de2d9be6 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:07.380524 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c53d085-13c6-45cd-8f46-f8891328cbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:07.381508 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53d085-13c6-45cd-8f46-f8891328cbd9 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:07.381937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53d085-13c6-45cd-8f46-f8891328cbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:07.382370 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53d085-13c6-45cd-8f46-f8891328cbd9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:07.395427 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c53d085-13c6-45cd-8f46-f8891328cbd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:07.396064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 68/127] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 10 22:34:07 2022] PUT /identity/v3/projects/2694f62f168849c0a5cd44bffc841cef/users/d91ed101f8354003baf64fb2ad037357/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:34:07.401442 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-3c53d085-13c6-45cd-8f46-f8891328cbd9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:07.401442 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:07.401442 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:07.401442 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:07.401442 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:07.401442 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:07.401442 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:07.401442 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:07.401442 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:07.401442 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:07.401442 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:07.401442 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:07.401442 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:07.401442 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:07.401442 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:07.401442 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:07.402417 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:07.402417 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:07.402417 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:07.402417 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:07.402417 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:07.402417 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:07.402417 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:07.402417 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:07.402417 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:07.402417 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:07.402417 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:07.402417 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:07.402417 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:07.402417 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:07.402417 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:07.402417 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:07.403182 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:07.403182 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:07.403182 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:07.403182 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:07.403182 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:07.403182 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:07.403182 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:07.403182 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:07.403182 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:07.403182 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:07.403182 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:07.403182 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:07.403759 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1952fd96-4046-4cb2-8f44-3990de2d9be6 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:07.405583 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 60/128] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:07.412211 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c53d085-13c6-45cd-8f46-f8891328cbd9 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:07.557460 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1952fd96-4046-4cb2-8f44-3990de2d9be6 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:07.596727 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:07.610512 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d90331-da09-4e97-936c-e1eb2987004b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:07.611434 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d90331-da09-4e97-936c-e1eb2987004b None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:07.611719 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d90331-da09-4e97-936c-e1eb2987004b None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:07.611999 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d90331-da09-4e97-936c-e1eb2987004b None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:07.612844 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d90331-da09-4e97-936c-e1eb2987004b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:07.613163 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d90331-da09-4e97-936c-e1eb2987004b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:07.614925 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d90331-da09-4e97-936c-e1eb2987004b None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:07.627716 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 69/129] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:34:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:07.628391 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d90331-da09-4e97-936c-e1eb2987004b None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:07.639591 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c53d085-13c6-45cd-8f46-f8891328cbd9 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:07.665361 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:07.691566 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05159f-fe2c-442b-b889-b0e440f36770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:07.692255 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05159f-fe2c-442b-b889-b0e440f36770 None admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:07.692546 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05159f-fe2c-442b-b889-b0e440f36770 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:07.692831 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05159f-fe2c-442b-b889-b0e440f36770 None admin] PATH_INFO: `/v3/projects/2694f62f168849c0a5cd44bffc841cef/users/6e08ec69331f4a57817e3de09ab9221e/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:07.694805 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05159f-fe2c-442b-b889-b0e440f36770 None admin] RBAC: Authorizing `identity:create_grant(project_id=2694f62f168849c0a5cd44bffc841cef, user_id=6e08ec69331f4a57817e3de09ab9221e, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:07.696279 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05159f-fe2c-442b-b889-b0e440f36770 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:07.716877 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 61/130] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 10 22:34:07 2022] PUT /identity/v3/projects/2694f62f168849c0a5cd44bffc841cef/users/6e08ec69331f4a57817e3de09ab9221e/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:34:07.718884 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05159f-fe2c-442b-b889-b0e440f36770 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:07.869224 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05159f-fe2c-442b-b889-b0e440f36770 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:07.884763 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d90331-da09-4e97-936c-e1eb2987004b None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:07.907214 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d6ed2-a305-4069-8077-5f0ee66086b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:07.907954 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d6ed2-a305-4069-8077-5f0ee66086b2 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:07.908245 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d6ed2-a305-4069-8077-5f0ee66086b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:07.908520 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d6ed2-a305-4069-8077-5f0ee66086b2 None admin] PATH_INFO: `/v3/users/6e08ec69331f4a57817e3de09ab9221e` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:07.909731 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769d6ed2-a305-4069-8077-5f0ee66086b2 None admin] RBAC: Authorizing `identity:get_user(user_id=6e08ec69331f4a57817e3de09ab9221e)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:07.911357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769d6ed2-a305-4069-8077-5f0ee66086b2 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:07.912988 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 62/131] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:34:07 2022] GET /identity/v3/users/6e08ec69331f4a57817e3de09ab9221e => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:07.918874 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d6ed2-a305-4069-8077-5f0ee66086b2 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:07.943595 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61254292-5f35-4d06-aa24-0626802f2d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:07.944221 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61254292-5f35-4d06-aa24-0626802f2d56 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:07.944510 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61254292-5f35-4d06-aa24-0626802f2d56 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:07.944845 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61254292-5f35-4d06-aa24-0626802f2d56 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:07.945615 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61254292-5f35-4d06-aa24-0626802f2d56 None admin] RBAC: Adding query filter params (name=member) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:07.946091 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61254292-5f35-4d06-aa24-0626802f2d56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:07.947784 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61254292-5f35-4d06-aa24-0626802f2d56 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:07.963438 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:07.963737 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 70/132] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 10 22:34:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:07.970174 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61254292-5f35-4d06-aa24-0626802f2d56 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:08.036397 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:08.048914 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d6ed2-a305-4069-8077-5f0ee66086b2 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:08.088042 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61254292-5f35-4d06-aa24-0626802f2d56 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:08.090591 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780359f-f869-4f82-b63f-7e76637a9b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:08.091474 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780359f-f869-4f82-b63f-7e76637a9b1f None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:08.091930 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780359f-f869-4f82-b63f-7e76637a9b1f None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:08.092382 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780359f-f869-4f82-b63f-7e76637a9b1f None admin] PATH_INFO: `/v3/projects/2694f62f168849c0a5cd44bffc841cef` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:08.093772 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2780359f-f869-4f82-b63f-7e76637a9b1f None admin] RBAC: Authorizing `identity:get_project(project_id=2694f62f168849c0a5cd44bffc841cef)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:08.095738 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2780359f-f869-4f82-b63f-7e76637a9b1f None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:08.097820 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 63/133] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:07 2022] GET /identity/v3/projects/2694f62f168849c0a5cd44bffc841cef => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:34:08.105389 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780359f-f869-4f82-b63f-7e76637a9b1f None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:08.148294 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac08bd-43b5-46bc-9907-0d93e174e560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:08.148864 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac08bd-43b5-46bc-9907-0d93e174e560 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:08.149147 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac08bd-43b5-46bc-9907-0d93e174e560 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:08.149449 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac08bd-43b5-46bc-9907-0d93e174e560 None admin] PATH_INFO: `/v3/users/6e08ec69331f4a57817e3de09ab9221e` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:08.151486 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac08bd-43b5-46bc-9907-0d93e174e560 None admin] RBAC: Authorizing `identity:get_user(user_id=6e08ec69331f4a57817e3de09ab9221e)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:08.152956 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac08bd-43b5-46bc-9907-0d93e174e560 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:08.154579 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 71/134] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:34:07 2022] GET /identity/v3/users/6e08ec69331f4a57817e3de09ab9221e => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:34:08.160753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac08bd-43b5-46bc-9907-0d93e174e560 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:08.287525 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac08bd-43b5-46bc-9907-0d93e174e560 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:08.312011 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:08.326617 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a13bf7-6d21-4f26-9bf6-6c1e1a33968a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:08.327625 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a13bf7-6d21-4f26-9bf6-6c1e1a33968a None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:08.328053 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a13bf7-6d21-4f26-9bf6-6c1e1a33968a None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:08.328497 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a13bf7-6d21-4f26-9bf6-6c1e1a33968a None admin] PATH_INFO: `/v3/projects/88da906d760243afb9b93d85aaabfa7e` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:08.331278 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a13bf7-6d21-4f26-9bf6-6c1e1a33968a None admin] RBAC: Authorizing `identity:get_project(project_id=88da906d760243afb9b93d85aaabfa7e)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:08.333487 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a13bf7-6d21-4f26-9bf6-6c1e1a33968a None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:08.335634 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 72/135] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:08 2022] GET /identity/v3/projects/88da906d760243afb9b93d85aaabfa7e => generated 312 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:08.341305 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780359f-f869-4f82-b63f-7e76637a9b1f None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:08.349725 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:08.351376 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a13bf7-6d21-4f26-9bf6-6c1e1a33968a None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:08.434051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680c090-48d9-4ff9-98ae-cf3b3f1ab465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:08.434708 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680c090-48d9-4ff9-98ae-cf3b3f1ab465 None admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:08.434995 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680c090-48d9-4ff9-98ae-cf3b3f1ab465 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:08.435275 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680c090-48d9-4ff9-98ae-cf3b3f1ab465 None admin] PATH_INFO: `/v3/projects/2694f62f168849c0a5cd44bffc841cef/users/6e08ec69331f4a57817e3de09ab9221e/roles/e22e4d2b3d06480bbcfa4d2ee3387c90` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:08.439811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3680c090-48d9-4ff9-98ae-cf3b3f1ab465 None admin] RBAC: Authorizing `identity:create_grant(project_id=2694f62f168849c0a5cd44bffc841cef, user_id=6e08ec69331f4a57817e3de09ab9221e, role_id=e22e4d2b3d06480bbcfa4d2ee3387c90)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:08.441329 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3680c090-48d9-4ff9-98ae-cf3b3f1ab465 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:08.464201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 64/136] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 10 22:34:08 2022] PUT /identity/v3/projects/2694f62f168849c0a5cd44bffc841cef/users/6e08ec69331f4a57817e3de09ab9221e/roles/e22e4d2b3d06480bbcfa4d2ee3387c90 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:34:08.494627 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a13bf7-6d21-4f26-9bf6-6c1e1a33968a None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:08.553164 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6184aa2-7b00-4f5a-9f42-762c362cb775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:08.554113 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6184aa2-7b00-4f5a-9f42-762c362cb775 None admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:08.554538 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6184aa2-7b00-4f5a-9f42-762c362cb775 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:08.554953 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6184aa2-7b00-4f5a-9f42-762c362cb775 None admin] PATH_INFO: `/v3/projects/88da906d760243afb9b93d85aaabfa7e/users/6e08ec69331f4a57817e3de09ab9221e/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:08.562910 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6184aa2-7b00-4f5a-9f42-762c362cb775 None admin] RBAC: Authorizing `identity:create_grant(project_id=88da906d760243afb9b93d85aaabfa7e, user_id=6e08ec69331f4a57817e3de09ab9221e, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:08.564912 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6184aa2-7b00-4f5a-9f42-762c362cb775 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:08.581594 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 73/137] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 10 22:34:08 2022] PUT /identity/v3/projects/88da906d760243afb9b93d85aaabfa7e/users/6e08ec69331f4a57817e3de09ab9221e/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:34:09.167905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df2c23-ded2-46e3-90c3-615e5701867d None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:09.168519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df2c23-ded2-46e3-90c3-615e5701867d None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:09.168519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df2c23-ded2-46e3-90c3-615e5701867d None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:09.170202 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 65/138] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:09.175788 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4f9cc-cf09-4d5c-a029-93849a46fa48 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:09.176238 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4f9cc-cf09-4d5c-a029-93849a46fa48 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:09.176544 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4f9cc-cf09-4d5c-a029-93849a46fa48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:09.196542 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-09a4f9cc-cf09-4d5c-a029-93849a46fa48 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:09.197799 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-09a4f9cc-cf09-4d5c-a029-93849a46fa48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:09.255188 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:09.283932 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 74/139] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:09.327164 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df2c23-ded2-46e3-90c3-615e5701867d None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:09.467963 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df2c23-ded2-46e3-90c3-615e5701867d None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:09.519431 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e227192-a3ac-4b8d-bfe2-876ec0cbf7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:09.520086 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e227192-a3ac-4b8d-bfe2-876ec0cbf7f5 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:09.520367 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e227192-a3ac-4b8d-bfe2-876ec0cbf7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:09.520642 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e227192-a3ac-4b8d-bfe2-876ec0cbf7f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:09.521797 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e227192-a3ac-4b8d-bfe2-876ec0cbf7f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:09.523255 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e227192-a3ac-4b8d-bfe2-876ec0cbf7f5 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:09.524840 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 66/140] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:34:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:09.530776 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a4f9cc-cf09-4d5c-a029-93849a46fa48 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:09.668018 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a4f9cc-cf09-4d5c-a029-93849a46fa48 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:09.717571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b9defc-4989-4790-93b6-24aa6d9699cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:09.718106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9defc-4989-4790-93b6-24aa6d9699cc None admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:09.718381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9defc-4989-4790-93b6-24aa6d9699cc None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:09.718648 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9defc-4989-4790-93b6-24aa6d9699cc None admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:09.719290 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b9defc-4989-4790-93b6-24aa6d9699cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:09.721025 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b9defc-4989-4790-93b6-24aa6d9699cc None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:09.751104 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 75/141] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 10 22:34:09 2022] POST /identity/v3/users => generated 283 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:34:10.031445 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9cf45-b317-42cb-923b-1869e509f02d None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:10.031789 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9cf45-b317-42cb-923b-1869e509f02d None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:10.031906 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9cf45-b317-42cb-923b-1869e509f02d None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:10.033727 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 67/142] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:10.040240 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc273af-c3f3-4a84-9b0b-45b912bea679 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:10.040526 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc273af-c3f3-4a84-9b0b-45b912bea679 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:10.040804 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc273af-c3f3-4a84-9b0b-45b912bea679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:10.067923 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-1dc273af-c3f3-4a84-9b0b-45b912bea679 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:10.069123 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-1dc273af-c3f3-4a84-9b0b-45b912bea679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:10.109856 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:10.162158 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 76/143] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:34:10.204353 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9cf45-b317-42cb-923b-1869e509f02d None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:10.246376 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:10.340405 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9cf45-b317-42cb-923b-1869e509f02d None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:10.380435 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67534c-98b8-49ec-8db7-0042358ac966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:10.381096 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67534c-98b8-49ec-8db7-0042358ac966 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:10.381407 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67534c-98b8-49ec-8db7-0042358ac966 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:10.381684 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67534c-98b8-49ec-8db7-0042358ac966 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:10.399551 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb67534c-98b8-49ec-8db7-0042358ac966 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:10.404336 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-bb67534c-98b8-49ec-8db7-0042358ac966 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:34:10.404336 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:10.404336 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:10.404336 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:10.404336 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:10.404336 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:10.404336 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:10.404336 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:10.404336 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:10.404336 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:10.404336 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:10.404336 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:10.404336 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:10.404336 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:10.404336 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:10.404336 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:10.405221 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:10.405221 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:10.405221 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:10.405221 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:10.405221 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:10.405221 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:10.405221 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:10.405221 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:10.405221 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:10.405221 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:10.405221 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:10.405221 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:10.405221 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:10.405221 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:10.405221 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:10.405221 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:10.406089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:10.406089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:10.406089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:10.406089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:10.406089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:10.406089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:10.406089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:10.406089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:10.406089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:10.406089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:10.406089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:34:10.406089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:10.406089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 68/144] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:10.411927 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc273af-c3f3-4a84-9b0b-45b912bea679 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:10.460240 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d416c6-905f-445c-b639-945004b13259 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:10.460573 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d416c6-905f-445c-b639-945004b13259 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:10.460808 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d416c6-905f-445c-b639-945004b13259 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:10.463097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 69/145] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:10.468303 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f4e47-8df6-4643-8947-8e428fbbfbfa None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:10.468580 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f4e47-8df6-4643-8947-8e428fbbfbfa None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:10.468853 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f4e47-8df6-4643-8947-8e428fbbfbfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:10.492930 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-998f4e47-8df6-4643-8947-8e428fbbfbfa None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:10.494625 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-998f4e47-8df6-4643-8947-8e428fbbfbfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:10.575340 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc273af-c3f3-4a84-9b0b-45b912bea679 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:10.602295 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 70/146] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:10.631284 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01bfa0-311e-4ee9-aa73-ba092e01148c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:10.632254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01bfa0-311e-4ee9-aa73-ba092e01148c None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:10.632769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01bfa0-311e-4ee9-aa73-ba092e01148c None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:10.633364 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01bfa0-311e-4ee9-aa73-ba092e01148c None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:10.634271 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f01bfa0-311e-4ee9-aa73-ba092e01148c None admin] RBAC: Adding query filter params (name=reader) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:10.634780 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f01bfa0-311e-4ee9-aa73-ba092e01148c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:10.636383 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f01bfa0-311e-4ee9-aa73-ba092e01148c None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:10.648879 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 77/147] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 10 22:34:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:10.654366 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f4e47-8df6-4643-8947-8e428fbbfbfa None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:10.681757 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01bfa0-311e-4ee9-aa73-ba092e01148c None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:10.686790 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:10.738938 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:10.800000 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f4e47-8df6-4643-8947-8e428fbbfbfa None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:10.832495 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01bfa0-311e-4ee9-aa73-ba092e01148c None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:10.855156 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f41165d-ecfa-4226-8a0a-3f2b2d2b3499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:10.855704 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41165d-ecfa-4226-8a0a-3f2b2d2b3499 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:10.855989 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41165d-ecfa-4226-8a0a-3f2b2d2b3499 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:10.856429 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41165d-ecfa-4226-8a0a-3f2b2d2b3499 None admin] PATH_INFO: `/v3/users/9ddd8e696aa24e88917377eb6e3199da` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:10.858451 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f41165d-ecfa-4226-8a0a-3f2b2d2b3499 None admin] RBAC: Authorizing `identity:get_user(user_id=9ddd8e696aa24e88917377eb6e3199da)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:10.859914 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f41165d-ecfa-4226-8a0a-3f2b2d2b3499 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:10.862249 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 71/148] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:34:10 2022] GET /identity/v3/users/9ddd8e696aa24e88917377eb6e3199da => generated 289 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:10.863554 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9b900-039a-465f-8dd8-9de386091773 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:10.863844 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9b900-039a-465f-8dd8-9de386091773 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:10.864418 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9b900-039a-465f-8dd8-9de386091773 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:10.870347 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 72/149] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:10.875542 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2971315-8a4d-45a8-8ffb-f1757163cc98 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:10.875832 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2971315-8a4d-45a8-8ffb-f1757163cc98 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:10.876110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2971315-8a4d-45a8-8ffb-f1757163cc98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:10.897909 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-c2971315-8a4d-45a8-8ffb-f1757163cc98 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:10.899104 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-c2971315-8a4d-45a8-8ffb-f1757163cc98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:10.901033 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2d539-649d-4ae8-96b1-3ba0a9c8e732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:10.901754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2d539-649d-4ae8-96b1-3ba0a9c8e732 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:10.902137 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2d539-649d-4ae8-96b1-3ba0a9c8e732 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:10.902415 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2d539-649d-4ae8-96b1-3ba0a9c8e732 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:10.930274 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a2d539-649d-4ae8-96b1-3ba0a9c8e732 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:10.937688 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-31a2d539-649d-4ae8-96b1-3ba0a9c8e732 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:34:10.937688 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:10.937688 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:10.937688 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:10.937688 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:10.937688 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:10.937688 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:10.937688 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:10.937688 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:10.937688 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:10.937688 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:10.937688 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:10.937688 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:10.937688 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:10.937688 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:10.937688 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:10.938524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:10.938524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:10.938524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:10.938524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:10.938524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:10.938524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:10.938524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:10.938524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:10.938524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:10.938524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:10.938524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:10.938524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:10.938524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:10.938524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:10.938524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:10.938524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:10.939514 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:10.939514 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:10.939514 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:10.939514 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:10.939514 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:10.939514 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:10.939514 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:10.939514 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:10.939514 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:10.939514 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:10.939514 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:34:10.939514 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:10.939514 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 78/150] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:34:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:34:10.939514 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2d539-649d-4ae8-96b1-3ba0a9c8e732 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:11.031200 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 73/151] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:11.031894 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2971315-8a4d-45a8-8ffb-f1757163cc98 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:11.080375 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:11.136417 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:11.159356 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2d539-649d-4ae8-96b1-3ba0a9c8e732 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:11.229272 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2971315-8a4d-45a8-8ffb-f1757163cc98 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:11.231361 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f48f9-92b0-4f1c-9aff-69ec64a91f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:11.232021 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f48f9-92b0-4f1c-9aff-69ec64a91f7a None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:11.232302 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f48f9-92b0-4f1c-9aff-69ec64a91f7a None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:11.232577 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f48f9-92b0-4f1c-9aff-69ec64a91f7a None admin] PATH_INFO: `/v3/projects/2694f62f168849c0a5cd44bffc841cef` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:11.233878 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f48f9-92b0-4f1c-9aff-69ec64a91f7a None admin] RBAC: Authorizing `identity:get_project(project_id=2694f62f168849c0a5cd44bffc841cef)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:11.235464 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f48f9-92b0-4f1c-9aff-69ec64a91f7a None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:11.239549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 79/152] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:10 2022] GET /identity/v3/projects/2694f62f168849c0a5cd44bffc841cef => generated 298 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:11.240383 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f48f9-92b0-4f1c-9aff-69ec64a91f7a None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:11.301912 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d7c7d-2875-4233-8d5c-3d8acd88a99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:11.302547 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d7c7d-2875-4233-8d5c-3d8acd88a99d None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:11.302827 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d7c7d-2875-4233-8d5c-3d8acd88a99d None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:11.303102 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d7c7d-2875-4233-8d5c-3d8acd88a99d None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:11.303782 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d7c7d-2875-4233-8d5c-3d8acd88a99d None admin] RBAC: Adding query filter params (name=admin) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:11.304088 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d7c7d-2875-4233-8d5c-3d8acd88a99d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:11.305518 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d7c7d-2875-4233-8d5c-3d8acd88a99d None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:11.325056 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 74/153] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:11.325934 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d7c7d-2875-4233-8d5c-3d8acd88a99d None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:11.465454 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f48f9-92b0-4f1c-9aff-69ec64a91f7a None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:11.507199 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:11.533153 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741474e-534e-443c-915d-48bad66735aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:11.534425 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741474e-534e-443c-915d-48bad66735aa None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:11.534785 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741474e-534e-443c-915d-48bad66735aa None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:11.535074 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741474e-534e-443c-915d-48bad66735aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:11.548273 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d7c7d-2875-4233-8d5c-3d8acd88a99d None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:11.559507 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a741474e-534e-443c-915d-48bad66735aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:11.565216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-a741474e-534e-443c-915d-48bad66735aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:11.565216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:11.565216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:11.565216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:11.565216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:11.565216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:11.565216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:11.565216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:11.565216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:11.565216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:11.565216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:11.565216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:11.565216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:11.565216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:11.565216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:11.565216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:11.566242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:11.566242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:11.566242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:11.566242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:11.566242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:11.566242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:11.566242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:11.566242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:11.566242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:11.566242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:11.566242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:11.566242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:11.566242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:11.566242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:11.566242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:11.566242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:11.567051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:11.567051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:11.567051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:11.567051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:11.567051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:11.567051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:11.567051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:11.567051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:11.567051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:11.567051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:11.567051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:11.567051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:11.567051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 80/154] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:11.567863 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741474e-534e-443c-915d-48bad66735aa None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:11.611045 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:11.633229 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b32e6-6482-4de9-b960-aab7427ae628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:11.633959 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b32e6-6482-4de9-b960-aab7427ae628 None admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:11.634244 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b32e6-6482-4de9-b960-aab7427ae628 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:11.634524 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b32e6-6482-4de9-b960-aab7427ae628 None admin] PATH_INFO: `/v3/projects/2694f62f168849c0a5cd44bffc841cef/users/9ddd8e696aa24e88917377eb6e3199da/roles/05a9201bdeee4f2ab18c648d5e6cb250` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:11.639374 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3b32e6-6482-4de9-b960-aab7427ae628 None admin] RBAC: Authorizing `identity:create_grant(project_id=2694f62f168849c0a5cd44bffc841cef, user_id=9ddd8e696aa24e88917377eb6e3199da, role_id=05a9201bdeee4f2ab18c648d5e6cb250)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:11.641063 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3b32e6-6482-4de9-b960-aab7427ae628 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:11.670870 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 75/155] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 10 22:34:11 2022] PUT /identity/v3/projects/2694f62f168849c0a5cd44bffc841cef/users/9ddd8e696aa24e88917377eb6e3199da/roles/05a9201bdeee4f2ab18c648d5e6cb250 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:34:11.671488 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b32e6-6482-4de9-b960-aab7427ae628 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:11.816936 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741474e-534e-443c-915d-48bad66735aa None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:11.888359 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3716047-e8c4-4bce-98ff-e3ba2c00e3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:11.889010 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3716047-e8c4-4bce-98ff-e3ba2c00e3fd None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:11.889312 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3716047-e8c4-4bce-98ff-e3ba2c00e3fd None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:11.889589 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3716047-e8c4-4bce-98ff-e3ba2c00e3fd None admin] PATH_INFO: `/v3/users/d91ed101f8354003baf64fb2ad037357` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:11.891584 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3716047-e8c4-4bce-98ff-e3ba2c00e3fd None admin] RBAC: Authorizing `identity:get_user(user_id=d91ed101f8354003baf64fb2ad037357)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:11.893641 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3716047-e8c4-4bce-98ff-e3ba2c00e3fd None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:11.895454 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 81/156] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:34:11 2022] GET /identity/v3/users/d91ed101f8354003baf64fb2ad037357 => generated 247 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:11.900156 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3800eab-a851-4a27-a877-856583c218af None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:11.900156 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3800eab-a851-4a27-a877-856583c218af None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:11.900156 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3800eab-a851-4a27-a877-856583c218af None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:11.900156 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 82/157] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:11.902823 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3800eab-a851-4a27-a877-856583c218af None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:11.909641 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b32e6-6482-4de9-b960-aab7427ae628 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:11.974021 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aa913a-6fec-46ae-ad0e-c01e61779603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:11.974704 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa913a-6fec-46ae-ad0e-c01e61779603 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:11.974984 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa913a-6fec-46ae-ad0e-c01e61779603 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:11.975259 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa913a-6fec-46ae-ad0e-c01e61779603 None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:11.975994 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aa913a-6fec-46ae-ad0e-c01e61779603 None admin] RBAC: Adding query filter params (name=member) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:11.976301 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aa913a-6fec-46ae-ad0e-c01e61779603 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:11.979311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aa913a-6fec-46ae-ad0e-c01e61779603 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:11.999702 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 76/158] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 10 22:34:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:12.000841 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee1b30-5355-4e9a-af55-089b021c3af6 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:12.001132 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee1b30-5355-4e9a-af55-089b021c3af6 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:12.002264 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee1b30-5355-4e9a-af55-089b021c3af6 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:12.003825 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 77/159] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:12.004806 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e57e3a-3b6b-47af-9a71-b69f6a0cc99a None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:12.005088 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e57e3a-3b6b-47af-9a71-b69f6a0cc99a None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:12.005388 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e57e3a-3b6b-47af-9a71-b69f6a0cc99a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:12.027335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:12.049696 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:12.049949 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-82e57e3a-3b6b-47af-9a71-b69f6a0cc99a None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:12.055688 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-82e57e3a-3b6b-47af-9a71-b69f6a0cc99a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:12.091116 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3800eab-a851-4a27-a877-856583c218af None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:12.144783 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 78/160] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:12.147454 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e57e3a-3b6b-47af-9a71-b69f6a0cc99a None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:12.152663 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da380c-14b8-45a8-95ea-d417d556a29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:12.153613 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da380c-14b8-45a8-95ea-d417d556a29c None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:12.154098 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da380c-14b8-45a8-95ea-d417d556a29c None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:12.154620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da380c-14b8-45a8-95ea-d417d556a29c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:12.341389 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44da380c-14b8-45a8-95ea-d417d556a29c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:12.342803 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44da380c-14b8-45a8-95ea-d417d556a29c None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:12.366619 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e57e3a-3b6b-47af-9a71-b69f6a0cc99a None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:12.453068 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac358b-3d92-4e2f-a810-863ae9c237b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:12.453787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac358b-3d92-4e2f-a810-863ae9c237b1 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:12.454066 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac358b-3d92-4e2f-a810-863ae9c237b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:12.454340 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac358b-3d92-4e2f-a810-863ae9c237b1 None admin] PATH_INFO: `/v3/users/6e08ec69331f4a57817e3de09ab9221e` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:12.457604 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac358b-3d92-4e2f-a810-863ae9c237b1 None admin] RBAC: Authorizing `identity:get_user(user_id=6e08ec69331f4a57817e3de09ab9221e)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:12.459065 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac358b-3d92-4e2f-a810-863ae9c237b1 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:12.460683 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 79/161] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:34:12 2022] GET /identity/v3/users/6e08ec69331f4a57817e3de09ab9221e => generated 275 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:12.465990 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fab757-a9db-49a9-9071-20b6df10615d None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:12.466403 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fab757-a9db-49a9-9071-20b6df10615d None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:12.466687 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fab757-a9db-49a9-9071-20b6df10615d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:12.477647 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:12.487092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-86fab757-a9db-49a9-9071-20b6df10615d None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:12.488300 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-86fab757-a9db-49a9-9071-20b6df10615d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:12.544794 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:12.564116 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 80/162] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:12.565950 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fab757-a9db-49a9-9071-20b6df10615d None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:12.573050 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 83/163] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:34:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 670 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:34:12.573916 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da380c-14b8-45a8-95ea-d417d556a29c None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:12.700962 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fab757-a9db-49a9-9071-20b6df10615d None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:12.739644 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1707f78-83a2-412e-bb2d-03e714e018d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:12.740683 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1707f78-83a2-412e-bb2d-03e714e018d6 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:12.741186 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1707f78-83a2-412e-bb2d-03e714e018d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:12.741695 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1707f78-83a2-412e-bb2d-03e714e018d6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:12.742581 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:12.759763 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1707f78-83a2-412e-bb2d-03e714e018d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:12.770169 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-d1707f78-83a2-412e-bb2d-03e714e018d6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:34:12.770169 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:12.770169 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:12.770169 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:12.770169 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:12.770169 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:12.770169 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:12.770169 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:12.770169 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:12.770169 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:12.770169 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:12.770169 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:12.770169 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:12.770169 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:12.770169 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:12.770169 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:12.771054 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:12.771054 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:12.771054 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:12.771054 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:12.771054 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:12.771054 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:12.771054 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:12.771054 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:12.771054 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:12.771054 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:12.771054 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:12.771054 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:12.771054 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:12.771054 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:12.771054 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:12.771054 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:12.771845 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:12.771845 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:12.771845 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:12.771845 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:12.771845 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:12.771845 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:12.771845 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:12.771845 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:12.771845 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:12.771845 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:12.771845 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:34:12.771845 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:12.774404 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 81/164] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 10 22:34:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:12.778290 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1707f78-83a2-412e-bb2d-03e714e018d6 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:12.800395 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da380c-14b8-45a8-95ea-d417d556a29c None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:12.929316 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a151baf4-87be-49d3-b0b9-4d594ea5152f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:12.930157 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151baf4-87be-49d3-b0b9-4d594ea5152f None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:12.930265 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151baf4-87be-49d3-b0b9-4d594ea5152f None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:12.930467 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151baf4-87be-49d3-b0b9-4d594ea5152f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:12.936215 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1707f78-83a2-412e-bb2d-03e714e018d6 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:12.980404 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c90cf56-3fec-49c1-bbe3-ba5f6ea7d28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:12.981631 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c90cf56-3fec-49c1-bbe3-ba5f6ea7d28b None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:12.982912 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c90cf56-3fec-49c1-bbe3-ba5f6ea7d28b None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:12.985411 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c90cf56-3fec-49c1-bbe3-ba5f6ea7d28b None admin] PATH_INFO: `/v3/projects/2694f62f168849c0a5cd44bffc841cef` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:12.988014 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c90cf56-3fec-49c1-bbe3-ba5f6ea7d28b None admin] RBAC: Authorizing `identity:get_project(project_id=2694f62f168849c0a5cd44bffc841cef)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:12.991090 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c90cf56-3fec-49c1-bbe3-ba5f6ea7d28b None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:12.995684 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 82/165] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:12 2022] GET /identity/v3/projects/2694f62f168849c0a5cd44bffc841cef => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:13.002996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c90cf56-3fec-49c1-bbe3-ba5f6ea7d28b None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:13.059396 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:13.079675 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a151baf4-87be-49d3-b0b9-4d594ea5152f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:13.082705 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a151baf4-87be-49d3-b0b9-4d594ea5152f None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:13.134263 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:13.219571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 84/166] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:34:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 646 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:34:13.220256 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a151baf4-87be-49d3-b0b9-4d594ea5152f None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:13.227411 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c90cf56-3fec-49c1-bbe3-ba5f6ea7d28b None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:13.299310 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a11eefe-fd9f-4f2b-ba20-07c68f53409d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:13.299938 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11eefe-fd9f-4f2b-ba20-07c68f53409d None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:13.300330 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11eefe-fd9f-4f2b-ba20-07c68f53409d None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:13.300605 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11eefe-fd9f-4f2b-ba20-07c68f53409d None admin] PATH_INFO: `/v3/roles/member` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:13.321673 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a11eefe-fd9f-4f2b-ba20-07c68f53409d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:13.327857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-8a11eefe-fd9f-4f2b-ba20-07c68f53409d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:13.327857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:13.327857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:13.327857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:13.327857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:13.327857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:13.327857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:13.327857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:13.327857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:13.327857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:13.327857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:13.327857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:13.327857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:13.327857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:13.327857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:13.327857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:13.328707 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:13.328707 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:13.328707 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:13.328707 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:13.328707 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:13.328707 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:13.328707 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:13.328707 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:13.328707 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:13.328707 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:13.328707 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:13.328707 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:13.328707 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:13.328707 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:13.328707 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:13.328707 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:13.329527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:13.329527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:13.329527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:13.329527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:13.329527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:13.329527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:13.329527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:13.329527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:13.329527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:13.329527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:13.329527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:13.329527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:13.332277 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 83/167] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:13.336649 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a11eefe-fd9f-4f2b-ba20-07c68f53409d None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:13.356243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:13.384549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a151baf4-87be-49d3-b0b9-4d594ea5152f None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:13.462102 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361bc77b-d366-435c-ab21-fe94b4fce564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:13.464396 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361bc77b-d366-435c-ab21-fe94b4fce564 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:13.464688 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361bc77b-d366-435c-ab21-fe94b4fce564 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:13.464963 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361bc77b-d366-435c-ab21-fe94b4fce564 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:13.465674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361bc77b-d366-435c-ab21-fe94b4fce564 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:13.466705 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361bc77b-d366-435c-ab21-fe94b4fce564 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:13.468104 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361bc77b-d366-435c-ab21-fe94b4fce564 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:13.487712 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 85/168] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:34:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:13.489124 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856b54a-d9d1-4ca8-a2e5-dc99e936d2a5 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:13.489446 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856b54a-d9d1-4ca8-a2e5-dc99e936d2a5 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:13.489715 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856b54a-d9d1-4ca8-a2e5-dc99e936d2a5 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:13.490937 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 86/169] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:13.491688 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0856b54a-d9d1-4ca8-a2e5-dc99e936d2a5 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:13.500523 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a11eefe-fd9f-4f2b-ba20-07c68f53409d None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:13.540463 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:13.570058 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81caaa00-3b08-43d5-a9ff-afae9458e4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:13.570898 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81caaa00-3b08-43d5-a9ff-afae9458e4d3 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:13.571320 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81caaa00-3b08-43d5-a9ff-afae9458e4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:13.571734 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81caaa00-3b08-43d5-a9ff-afae9458e4d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:13.572677 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81caaa00-3b08-43d5-a9ff-afae9458e4d3 None admin] RBAC: Adding query filter params (role.id=6641e95fdc1f4c098ebc372bb692d227, scope.project.id=2694f62f168849c0a5cd44bffc841cef, user.id=d91ed101f8354003baf64fb2ad037357) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:13.573123 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81caaa00-3b08-43d5-a9ff-afae9458e4d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:13.579898 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81caaa00-3b08-43d5-a9ff-afae9458e4d3 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:13.594727 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 84/170] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:34:13 2022] GET /identity/v3/role_assignments?user.id=d91ed101f8354003baf64fb2ad037357&scope.project.id=2694f62f168849c0a5cd44bffc841cef&role.id=6641e95fdc1f4c098ebc372bb692d227 => generated 608 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:13.595718 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81caaa00-3b08-43d5-a9ff-afae9458e4d3 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:13.613307 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0856b54a-d9d1-4ca8-a2e5-dc99e936d2a5 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:13.679339 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c337a0c-6b14-48df-ad9f-1a5491e94fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:13.679339 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c337a0c-6b14-48df-ad9f-1a5491e94fc6 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:13.679339 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c337a0c-6b14-48df-ad9f-1a5491e94fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:13.679339 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c337a0c-6b14-48df-ad9f-1a5491e94fc6 None admin] PATH_INFO: `/v3/projects/2694f62f168849c0a5cd44bffc841cef` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:13.683117 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c337a0c-6b14-48df-ad9f-1a5491e94fc6 None admin] RBAC: Authorizing `identity:get_project(project_id=2694f62f168849c0a5cd44bffc841cef)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:13.685032 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c337a0c-6b14-48df-ad9f-1a5491e94fc6 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:13.687057 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 87/171] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:13 2022] GET /identity/v3/projects/2694f62f168849c0a5cd44bffc841cef => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:13.688521 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b3bcc-c291-4cd1-8609-da929c3df0e8 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:13.688876 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b3bcc-c291-4cd1-8609-da929c3df0e8 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:13.692112 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b3bcc-c291-4cd1-8609-da929c3df0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:13.727163 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-b39b3bcc-c291-4cd1-8609-da929c3df0e8 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:13.728652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-b39b3bcc-c291-4cd1-8609-da929c3df0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:13.745640 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:13.816953 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81caaa00-3b08-43d5-a9ff-afae9458e4d3 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:13.905680 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97b21a-0cfd-4711-ac0c-a63752dac676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:13.906310 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97b21a-0cfd-4711-ac0c-a63752dac676 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:13.906635 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97b21a-0cfd-4711-ac0c-a63752dac676 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:13.906911 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97b21a-0cfd-4711-ac0c-a63752dac676 None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:13.907582 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97b21a-0cfd-4711-ac0c-a63752dac676 None admin] RBAC: Adding query filter params (name=member) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:13.907878 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97b21a-0cfd-4711-ac0c-a63752dac676 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:13.909310 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97b21a-0cfd-4711-ac0c-a63752dac676 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:13.919957 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 85/172] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 10 22:34:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:13.920793 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97b21a-0cfd-4711-ac0c-a63752dac676 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:13.931417 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 88/173] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:13.932286 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b3bcc-c291-4cd1-8609-da929c3df0e8 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:14.013482 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:14.120220 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97b21a-0cfd-4711-ac0c-a63752dac676 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:14.135689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b3bcc-c291-4cd1-8609-da929c3df0e8 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:14.178235 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd2611-a030-4dd3-84c8-dab74c8b06e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:14.179063 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd2611-a030-4dd3-84c8-dab74c8b06e7 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:14.179355 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd2611-a030-4dd3-84c8-dab74c8b06e7 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:14.179630 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd2611-a030-4dd3-84c8-dab74c8b06e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:14.180359 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd2611-a030-4dd3-84c8-dab74c8b06e7 None admin] RBAC: Adding query filter params (role.id=1085473c6e534f87812b05c7014f77c2, scope.project.id=2694f62f168849c0a5cd44bffc841cef, user.id=6e08ec69331f4a57817e3de09ab9221e) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:14.180668 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd2611-a030-4dd3-84c8-dab74c8b06e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:14.183836 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd2611-a030-4dd3-84c8-dab74c8b06e7 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:14.191300 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 89/174] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:34:13 2022] GET /identity/v3/role_assignments?user.id=6e08ec69331f4a57817e3de09ab9221e&scope.project.id=2694f62f168849c0a5cd44bffc841cef&role.id=1085473c6e534f87812b05c7014f77c2 => generated 608 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:14.198685 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd2611-a030-4dd3-84c8-dab74c8b06e7 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:14.229674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:14.250748 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c97a935-b184-4318-b7c8-1848f959754a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:14.251405 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97a935-b184-4318-b7c8-1848f959754a None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:14.251701 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97a935-b184-4318-b7c8-1848f959754a None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:14.251978 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97a935-b184-4318-b7c8-1848f959754a None admin] PATH_INFO: `/v3/users/6e08ec69331f4a57817e3de09ab9221e` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:14.255150 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c97a935-b184-4318-b7c8-1848f959754a None admin] RBAC: Authorizing `identity:get_user(user_id=6e08ec69331f4a57817e3de09ab9221e)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:14.256653 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c97a935-b184-4318-b7c8-1848f959754a None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:14.258750 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 86/175] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:34:13 2022] GET /identity/v3/users/6e08ec69331f4a57817e3de09ab9221e => generated 275 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:14.259668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c97a935-b184-4318-b7c8-1848f959754a None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:14.362684 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd2611-a030-4dd3-84c8-dab74c8b06e7 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:14.430169 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ce5f0-a615-4da3-ae87-d50797715053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:14.430761 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ce5f0-a615-4da3-ae87-d50797715053 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:14.431503 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ce5f0-a615-4da3-ae87-d50797715053 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:14.431503 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ce5f0-a615-4da3-ae87-d50797715053 None admin] PATH_INFO: `/v3/users/6e08ec69331f4a57817e3de09ab9221e` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:14.432546 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ce5f0-a615-4da3-ae87-d50797715053 None admin] RBAC: Authorizing `identity:get_user(user_id=6e08ec69331f4a57817e3de09ab9221e)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:14.434049 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ce5f0-a615-4da3-ae87-d50797715053 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:14.435661 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 90/176] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:34:14 2022] GET /identity/v3/users/6e08ec69331f4a57817e3de09ab9221e => generated 275 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:14.436433 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ce5f0-a615-4da3-ae87-d50797715053 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:14.479496 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c97a935-b184-4318-b7c8-1848f959754a None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:14.514895 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:14.546097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5d792-cedf-41c7-bcfb-d167e0759107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:14.546712 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5d792-cedf-41c7-bcfb-d167e0759107 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:14.546994 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5d792-cedf-41c7-bcfb-d167e0759107 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:14.547264 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5d792-cedf-41c7-bcfb-d167e0759107 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:14.565765 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:14.568874 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c5d792-cedf-41c7-bcfb-d167e0759107 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:14.575917 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ce5f0-a615-4da3-ae87-d50797715053 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:14.577009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-41c5d792-cedf-41c7-bcfb-d167e0759107 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:34:14.577009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:14.577009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:14.577009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:14.577009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:14.577009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:14.577009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:14.577009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:14.577009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:14.577009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:14.577009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:14.577009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:14.577009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:14.577009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:14.577009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:14.577009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:14.578003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:14.578003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:14.578003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:14.578003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:14.578003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:14.578003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:14.578003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:14.578003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:14.578003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:14.578003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:14.578003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:14.578003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:14.578003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:14.578003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:14.578003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:14.578003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:14.578937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:14.578937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:14.578937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:14.578937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:14.578937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:14.578937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:14.578937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:14.578937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:14.578937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:14.578937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:14.578937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:34:14.578937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:14.584802 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 87/177] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:34:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:14.585201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5d792-cedf-41c7-bcfb-d167e0759107 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:14.627243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be17e21-370c-47d2-8a1e-d649765b9590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:14.628119 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be17e21-370c-47d2-8a1e-d649765b9590 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:14.628589 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be17e21-370c-47d2-8a1e-d649765b9590 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:14.629061 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be17e21-370c-47d2-8a1e-d649765b9590 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:14.721087 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5d792-cedf-41c7-bcfb-d167e0759107 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:14.722391 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be17e21-370c-47d2-8a1e-d649765b9590 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:14.725273 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be17e21-370c-47d2-8a1e-d649765b9590 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:14.759039 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc1062a-f9d1-4d07-87ba-7617b385069e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:14.759656 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1062a-f9d1-4d07-87ba-7617b385069e None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:14.759933 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1062a-f9d1-4d07-87ba-7617b385069e None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:14.760203 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1062a-f9d1-4d07-87ba-7617b385069e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:14.805128 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:14.809448 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:14.829461 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 91/178] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:34:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:34:14.830783 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe858c-874b-451a-94af-ac1fd36cac9e None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:14.831636 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe858c-874b-451a-94af-ac1fd36cac9e None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:14.832159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe858c-874b-451a-94af-ac1fd36cac9e None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:14.833733 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 92/179] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:14.836699 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc1062a-f9d1-4d07-87ba-7617b385069e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:14.838169 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe858c-874b-451a-94af-ac1fd36cac9e None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:14.838281 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc1062a-f9d1-4d07-87ba-7617b385069e None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:14.936281 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 88/180] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:34:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:34:14.937573 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f396f347-ed6b-4e9e-a0e0-08c9564781e3 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:14.937858 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f396f347-ed6b-4e9e-a0e0-08c9564781e3 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:14.938138 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f396f347-ed6b-4e9e-a0e0-08c9564781e3 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:14.939378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 89/181] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:14.941397 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2955e-48cf-4697-b710-aad68496a367 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:14.941397 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2955e-48cf-4697-b710-aad68496a367 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:14.941397 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2955e-48cf-4697-b710-aad68496a367 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:14.942291 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 90/182] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:14.942854 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2955e-48cf-4697-b710-aad68496a367 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:14.945395 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe858c-874b-451a-94af-ac1fd36cac9e None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:14.982094 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3128daf-81a9-484e-93b2-69cf59219449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:14.982620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3128daf-81a9-484e-93b2-69cf59219449 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:14.982895 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3128daf-81a9-484e-93b2-69cf59219449 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:14.983165 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3128daf-81a9-484e-93b2-69cf59219449 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:14.983817 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3128daf-81a9-484e-93b2-69cf59219449 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:14.984114 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3128daf-81a9-484e-93b2-69cf59219449 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:14.985532 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3128daf-81a9-484e-93b2-69cf59219449 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:14.994027 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 93/183] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:14.995197 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce78076-f3b2-4958-85cf-da45e342448f None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:14.995514 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce78076-f3b2-4958-85cf-da45e342448f None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:14.995812 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce78076-f3b2-4958-85cf-da45e342448f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:15.012864 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-0ce78076-f3b2-4958-85cf-da45e342448f None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:15.014088 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-0ce78076-f3b2-4958-85cf-da45e342448f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:15.024121 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:15.044559 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:15.053160 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2955e-48cf-4697-b710-aad68496a367 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:15.083034 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 94/184] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:15.083875 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce78076-f3b2-4958-85cf-da45e342448f None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:15.093833 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6d2686-d79b-41c7-86c8-771393680eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:15.094506 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d2686-d79b-41c7-86c8-771393680eb7 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:15.094804 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d2686-d79b-41c7-86c8-771393680eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:15.095074 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d2686-d79b-41c7-86c8-771393680eb7 None admin] PATH_INFO: `/v3/projects/2694f62f168849c0a5cd44bffc841cef` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:15.096204 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6d2686-d79b-41c7-86c8-771393680eb7 None admin] RBAC: Authorizing `identity:get_project(project_id=2694f62f168849c0a5cd44bffc841cef)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:15.100077 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6d2686-d79b-41c7-86c8-771393680eb7 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:15.103457 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 91/185] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:14 2022] GET /identity/v3/projects/2694f62f168849c0a5cd44bffc841cef => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:15.108654 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f39cbb-1556-471d-b440-932a88df2e59 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:15.108654 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f39cbb-1556-471d-b440-932a88df2e59 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:15.108654 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f39cbb-1556-471d-b440-932a88df2e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:15.123717 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-36f39cbb-1556-471d-b440-932a88df2e59 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:15.124931 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-36f39cbb-1556-471d-b440-932a88df2e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:15.192149 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 92/186] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:15.193608 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa085483-f54b-48ec-b65b-844e1b9a211e None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:15.193895 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa085483-f54b-48ec-b65b-844e1b9a211e None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:15.194172 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa085483-f54b-48ec-b65b-844e1b9a211e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:15.205875 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce78076-f3b2-4958-85cf-da45e342448f None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:15.213526 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-aa085483-f54b-48ec-b65b-844e1b9a211e None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:15.215006 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-aa085483-f54b-48ec-b65b-844e1b9a211e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:15.244832 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df1e78-0b67-40ae-9b99-430c0be2cfa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:15.245444 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df1e78-0b67-40ae-9b99-430c0be2cfa1 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:15.245724 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df1e78-0b67-40ae-9b99-430c0be2cfa1 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:15.245995 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df1e78-0b67-40ae-9b99-430c0be2cfa1 None admin] PATH_INFO: `/v3/projects/88da906d760243afb9b93d85aaabfa7e` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:15.247118 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46df1e78-0b67-40ae-9b99-430c0be2cfa1 None admin] RBAC: Authorizing `identity:get_project(project_id=88da906d760243afb9b93d85aaabfa7e)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:15.248560 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46df1e78-0b67-40ae-9b99-430c0be2cfa1 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:15.250400 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 95/187] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:15 2022] GET /identity/v3/projects/88da906d760243afb9b93d85aaabfa7e => generated 312 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:15.251161 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df1e78-0b67-40ae-9b99-430c0be2cfa1 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:15.282703 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 93/188] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:15.283840 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa085483-f54b-48ec-b65b-844e1b9a211e None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:15.286676 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:15.296257 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:15.362305 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df1e78-0b67-40ae-9b99-430c0be2cfa1 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:15.385644 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa085483-f54b-48ec-b65b-844e1b9a211e None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:15.396712 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce39d61-3191-421f-9de6-f7b993f74668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:15.397398 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce39d61-3191-421f-9de6-f7b993f74668 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:15.397678 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce39d61-3191-421f-9de6-f7b993f74668 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:15.397954 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce39d61-3191-421f-9de6-f7b993f74668 None admin] PATH_INFO: `/v3/users/fa2b48749a564275b6ac6bed107a81a5` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:15.412456 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce39d61-3191-421f-9de6-f7b993f74668 None admin] RBAC: Authorizing `identity:get_user(user_id=fa2b48749a564275b6ac6bed107a81a5)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:15.413971 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce39d61-3191-421f-9de6-f7b993f74668 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:15.415838 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 96/189] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:34:15 2022] GET /identity/v3/users/fa2b48749a564275b6ac6bed107a81a5 => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:15.416401 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce39d61-3191-421f-9de6-f7b993f74668 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:15.424792 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f68eb-8665-4168-a44c-81b323b75da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:15.428110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f68eb-8665-4168-a44c-81b323b75da8 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:15.428110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f68eb-8665-4168-a44c-81b323b75da8 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:15.428110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f68eb-8665-4168-a44c-81b323b75da8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:15.436384 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f68eb-8665-4168-a44c-81b323b75da8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:15.441097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-788f68eb-8665-4168-a44c-81b323b75da8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:34:15.441097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:15.441097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:15.441097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:15.441097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:15.441097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:15.441097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:15.441097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:15.441097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:15.441097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:15.441097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:15.441097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:15.441097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:15.441097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:15.441097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:15.441097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:15.442095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:15.442095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:15.442095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:15.442095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:15.442095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:15.442095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:15.442095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:15.442095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:15.442095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:15.442095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:15.442095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:15.442095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:15.442095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:15.442095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:15.442095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:15.442095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:15.447837 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:15.447837 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:15.447837 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:15.447837 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:15.447837 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:15.447837 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:15.447837 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:15.447837 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:15.447837 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:15.447837 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:15.447837 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:34:15.447837 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:15.447837 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 94/190] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 10 22:34:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:15.447837 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f68eb-8665-4168-a44c-81b323b75da8 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:15.509534 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:15.513664 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:15.521555 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce39d61-3191-421f-9de6-f7b993f74668 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:15.550896 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f68eb-8665-4168-a44c-81b323b75da8 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:15.556635 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34493715-edaa-429c-951a-bd27ee90b3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:15.557247 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34493715-edaa-429c-951a-bd27ee90b3bf None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:15.557563 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34493715-edaa-429c-951a-bd27ee90b3bf None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:15.557836 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34493715-edaa-429c-951a-bd27ee90b3bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:15.558984 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34493715-edaa-429c-951a-bd27ee90b3bf None admin] RBAC: Adding query filter params (role.id=e22e4d2b3d06480bbcfa4d2ee3387c90, scope.project.id=2694f62f168849c0a5cd44bffc841cef, user.id=6e08ec69331f4a57817e3de09ab9221e) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:15.559294 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34493715-edaa-429c-951a-bd27ee90b3bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:15.561121 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34493715-edaa-429c-951a-bd27ee90b3bf None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:15.568204 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 97/191] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:34:15 2022] GET /identity/v3/role_assignments?user.id=6e08ec69331f4a57817e3de09ab9221e&scope.project.id=2694f62f168849c0a5cd44bffc841cef&role.id=e22e4d2b3d06480bbcfa4d2ee3387c90 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:15.568997 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34493715-edaa-429c-951a-bd27ee90b3bf None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:15.585901 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc3d55-6a35-44bb-bd21-acbf4cfaeb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:15.586563 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc3d55-6a35-44bb-bd21-acbf4cfaeb8e None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:15.586851 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc3d55-6a35-44bb-bd21-acbf4cfaeb8e None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:15.587119 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc3d55-6a35-44bb-bd21-acbf4cfaeb8e None admin] PATH_INFO: `/v3/domains/default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:15.588334 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffc3d55-6a35-44bb-bd21-acbf4cfaeb8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:15.590142 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffc3d55-6a35-44bb-bd21-acbf4cfaeb8e None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:15.591818 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 95/192] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:34:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:15.592627 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc3d55-6a35-44bb-bd21-acbf4cfaeb8e None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:15.692103 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34493715-edaa-429c-951a-bd27ee90b3bf None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:15.708662 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc3d55-6a35-44bb-bd21-acbf4cfaeb8e None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:15.729563 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ad613-a3ca-46d0-bc5c-d4e35dfba90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:15.731016 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ad613-a3ca-46d0-bc5c-d4e35dfba90e None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:15.731016 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ad613-a3ca-46d0-bc5c-d4e35dfba90e None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:15.731016 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ad613-a3ca-46d0-bc5c-d4e35dfba90e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:15.731468 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ad613-a3ca-46d0-bc5c-d4e35dfba90e None admin] RBAC: Adding query filter params (role.id=1085473c6e534f87812b05c7014f77c2, scope.project.id=88da906d760243afb9b93d85aaabfa7e, user.id=6e08ec69331f4a57817e3de09ab9221e) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:15.731790 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ad613-a3ca-46d0-bc5c-d4e35dfba90e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:15.733242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ad613-a3ca-46d0-bc5c-d4e35dfba90e None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:15.740651 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 98/193] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:34:15 2022] GET /identity/v3/role_assignments?user.id=6e08ec69331f4a57817e3de09ab9221e&scope.project.id=88da906d760243afb9b93d85aaabfa7e&role.id=1085473c6e534f87812b05c7014f77c2 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:15.745315 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ad613-a3ca-46d0-bc5c-d4e35dfba90e None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:15.745693 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:15.754418 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29caec6-629a-45ef-a24a-849a3b651c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:15.755112 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29caec6-629a-45ef-a24a-849a3b651c77 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:15.755392 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29caec6-629a-45ef-a24a-849a3b651c77 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:15.755663 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29caec6-629a-45ef-a24a-849a3b651c77 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:15.769524 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29caec6-629a-45ef-a24a-849a3b651c77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:15.772126 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:15.775487 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-f29caec6-629a-45ef-a24a-849a3b651c77 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:34:15.775487 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:15.775487 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:15.775487 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:15.775487 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:15.775487 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:15.775487 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:15.775487 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:15.775487 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:15.775487 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:15.775487 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:15.775487 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:15.775487 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:15.775487 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:15.775487 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:15.775487 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:15.776397 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:15.776397 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:15.776397 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:15.776397 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:15.776397 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:15.776397 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:15.776397 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:15.776397 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:15.776397 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:15.776397 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:15.776397 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:15.776397 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:15.776397 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:15.776397 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:15.776397 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:15.776397 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:15.777175 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:15.777175 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:15.777175 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:15.777175 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:15.777175 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:15.777175 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:15.777175 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:15.777175 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:15.777175 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:15.777175 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:15.777175 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:34:15.777175 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:15.777175 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 96/194] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:15.778507 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29caec6-629a-45ef-a24a-849a3b651c77 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:15.896593 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ad613-a3ca-46d0-bc5c-d4e35dfba90e None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:15.915052 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29caec6-629a-45ef-a24a-849a3b651c77 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:15.939553 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831f0f7-4a77-4f73-acd4-4290f6b8b049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:15.940417 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831f0f7-4a77-4f73-acd4-4290f6b8b049 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:15.940725 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831f0f7-4a77-4f73-acd4-4290f6b8b049 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:15.941109 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831f0f7-4a77-4f73-acd4-4290f6b8b049 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:15.954558 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e85317-98ee-489e-8d95-6babff4b296b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:15.955178 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e85317-98ee-489e-8d95-6babff4b296b None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:15.955456 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e85317-98ee-489e-8d95-6babff4b296b None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:15.955729 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e85317-98ee-489e-8d95-6babff4b296b None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:15.956965 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e85317-98ee-489e-8d95-6babff4b296b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:15.957301 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e85317-98ee-489e-8d95-6babff4b296b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:15.958777 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e85317-98ee-489e-8d95-6babff4b296b None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:15.971736 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 97/195] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:34:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:15.971736 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e85317-98ee-489e-8d95-6babff4b296b None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:16.023124 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8831f0f7-4a77-4f73-acd4-4290f6b8b049 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:16.024544 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8831f0f7-4a77-4f73-acd4-4290f6b8b049 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:16.027733 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:16.047549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:16.077037 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e85317-98ee-489e-8d95-6babff4b296b None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:16.120649 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceefcbcf-c48f-4400-b01d-32c0b68b487a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:16.121272 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceefcbcf-c48f-4400-b01d-32c0b68b487a None admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:16.121580 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceefcbcf-c48f-4400-b01d-32c0b68b487a None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:16.121850 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceefcbcf-c48f-4400-b01d-32c0b68b487a None admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:16.122501 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceefcbcf-c48f-4400-b01d-32c0b68b487a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:16.123960 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceefcbcf-c48f-4400-b01d-32c0b68b487a None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:16.129957 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 99/196] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:34:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:34:16.129957 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831f0f7-4a77-4f73-acd4-4290f6b8b049 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:16.147561 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 98/197] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 10 22:34:15 2022] POST /identity/v3/users => generated 297 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:34:16.148401 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceefcbcf-c48f-4400-b01d-32c0b68b487a None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:16.247152 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831f0f7-4a77-4f73-acd4-4290f6b8b049 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:16.271029 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceefcbcf-c48f-4400-b01d-32c0b68b487a None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:16.275024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:16.284842 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:16.288940 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b320853-e677-4aff-9a73-3bde31200762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:16.289745 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b320853-e677-4aff-9a73-3bde31200762 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:16.290105 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b320853-e677-4aff-9a73-3bde31200762 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:16.290439 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b320853-e677-4aff-9a73-3bde31200762 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:16.290895 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b320853-e677-4aff-9a73-3bde31200762 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:16.291198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b320853-e677-4aff-9a73-3bde31200762 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:16.292608 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b320853-e677-4aff-9a73-3bde31200762 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:16.302065 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 100/198] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 10 22:34:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:16.302896 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b320853-e677-4aff-9a73-3bde31200762 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:16.325533 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d448e7-9f2a-4e05-b066-ac42f1059492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:16.326462 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d448e7-9f2a-4e05-b066-ac42f1059492 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:16.326746 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d448e7-9f2a-4e05-b066-ac42f1059492 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:16.327021 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d448e7-9f2a-4e05-b066-ac42f1059492 None admin] PATH_INFO: `/v3/users/fa2b48749a564275b6ac6bed107a81a5` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:16.328341 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d448e7-9f2a-4e05-b066-ac42f1059492 None admin] RBAC: Authorizing `identity:get_user(user_id=fa2b48749a564275b6ac6bed107a81a5)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:16.330250 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d448e7-9f2a-4e05-b066-ac42f1059492 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:16.332448 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 99/199] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:34:16 2022] GET /identity/v3/users/fa2b48749a564275b6ac6bed107a81a5 => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:16.333226 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d448e7-9f2a-4e05-b066-ac42f1059492 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:16.435860 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b320853-e677-4aff-9a73-3bde31200762 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:16.448467 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d448e7-9f2a-4e05-b066-ac42f1059492 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:16.481929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ac12b-b353-47f6-8076-00e2606c9c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:16.482572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ac12b-b353-47f6-8076-00e2606c9c7f None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:16.482854 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ac12b-b353-47f6-8076-00e2606c9c7f None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:16.483131 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ac12b-b353-47f6-8076-00e2606c9c7f None admin] PATH_INFO: `/v3/projects/d6cc740b2ba549809a3b7f863d43517c` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:16.484257 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875ac12b-b353-47f6-8076-00e2606c9c7f None admin] RBAC: Authorizing `identity:get_project(project_id=d6cc740b2ba549809a3b7f863d43517c)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:16.485654 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aafad26-ed06-4582-8c91-14e1900cf9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:16.485776 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875ac12b-b353-47f6-8076-00e2606c9c7f None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:16.486409 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafad26-ed06-4582-8c91-14e1900cf9c6 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:16.486692 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafad26-ed06-4582-8c91-14e1900cf9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:16.486961 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafad26-ed06-4582-8c91-14e1900cf9c6 None admin] PATH_INFO: `/v3/users/9ddd8e696aa24e88917377eb6e3199da` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:16.487666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 101/200] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:16 2022] GET /identity/v3/projects/d6cc740b2ba549809a3b7f863d43517c => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:16.488208 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ac12b-b353-47f6-8076-00e2606c9c7f None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:16.489215 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aafad26-ed06-4582-8c91-14e1900cf9c6 None admin] RBAC: Authorizing `identity:get_user(user_id=9ddd8e696aa24e88917377eb6e3199da)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:16.490831 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aafad26-ed06-4582-8c91-14e1900cf9c6 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:16.492382 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 100/201] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:34:16 2022] GET /identity/v3/users/9ddd8e696aa24e88917377eb6e3199da => generated 289 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:16.498033 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aafad26-ed06-4582-8c91-14e1900cf9c6 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:16.550306 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:16.555597 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:16.617432 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aafad26-ed06-4582-8c91-14e1900cf9c6 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:16.628129 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ac12b-b353-47f6-8076-00e2606c9c7f None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:16.663573 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef03bb-66a2-467f-9ce7-542b0d7d76dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:16.664662 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef03bb-66a2-467f-9ce7-542b0d7d76dc None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:16.665229 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef03bb-66a2-467f-9ce7-542b0d7d76dc None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:16.665890 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef03bb-66a2-467f-9ce7-542b0d7d76dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:16.666922 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ef03bb-66a2-467f-9ce7-542b0d7d76dc None admin] RBAC: Adding query filter params (role.id=6641e95fdc1f4c098ebc372bb692d227, scope.project.id=d6cc740b2ba549809a3b7f863d43517c, user.id=fa2b48749a564275b6ac6bed107a81a5) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:16.667077 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15d99e-c403-4b1b-a4f4-29acc46becc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:16.667569 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ef03bb-66a2-467f-9ce7-542b0d7d76dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:16.667835 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15d99e-c403-4b1b-a4f4-29acc46becc3 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:16.668183 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15d99e-c403-4b1b-a4f4-29acc46becc3 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:16.668528 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15d99e-c403-4b1b-a4f4-29acc46becc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:16.669921 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ef03bb-66a2-467f-9ce7-542b0d7d76dc None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:16.683180 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 101/202] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:34:16 2022] GET /identity/v3/role_assignments?user.id=fa2b48749a564275b6ac6bed107a81a5&scope.project.id=d6cc740b2ba549809a3b7f863d43517c&role.id=6641e95fdc1f4c098ebc372bb692d227 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:16.683180 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef03bb-66a2-467f-9ce7-542b0d7d76dc None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:16.788688 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15d99e-c403-4b1b-a4f4-29acc46becc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:16.790225 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15d99e-c403-4b1b-a4f4-29acc46becc3 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:16.865670 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:16.903337 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef03bb-66a2-467f-9ce7-542b0d7d76dc None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:16.914114 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:16.932910 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 102/203] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:34:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:34:16.938828 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15d99e-c403-4b1b-a4f4-29acc46becc3 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:16.971709 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0305b71-4bc3-4644-93bc-278d8a0eb713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:16.973348 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0305b71-4bc3-4644-93bc-278d8a0eb713 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:16.973840 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0305b71-4bc3-4644-93bc-278d8a0eb713 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:16.974258 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0305b71-4bc3-4644-93bc-278d8a0eb713 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:17.079414 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15d99e-c403-4b1b-a4f4-29acc46becc3 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:17.116963 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539d010-59df-43e4-8b52-e3cbe43522d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:17.117870 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539d010-59df-43e4-8b52-e3cbe43522d8 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:17.118291 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539d010-59df-43e4-8b52-e3cbe43522d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:17.118763 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539d010-59df-43e4-8b52-e3cbe43522d8 None admin] PATH_INFO: `/v3/projects/d6cc740b2ba549809a3b7f863d43517c` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:17.120069 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a539d010-59df-43e4-8b52-e3cbe43522d8 None admin] RBAC: Authorizing `identity:get_project(project_id=d6cc740b2ba549809a3b7f863d43517c)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:17.121715 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a539d010-59df-43e4-8b52-e3cbe43522d8 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:17.123793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 103/204] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:16 2022] GET /identity/v3/projects/d6cc740b2ba549809a3b7f863d43517c => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:34:17.130507 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539d010-59df-43e4-8b52-e3cbe43522d8 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:17.150521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0305b71-4bc3-4644-93bc-278d8a0eb713 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:17.153154 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0305b71-4bc3-4644-93bc-278d8a0eb713 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:17.154874 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:17.241908 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539d010-59df-43e4-8b52-e3cbe43522d8 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:17.279588 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0814ab81-ce9a-4faf-9be6-f0a90c416286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:17.280632 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814ab81-ce9a-4faf-9be6-f0a90c416286 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:17.281132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814ab81-ce9a-4faf-9be6-f0a90c416286 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:17.281639 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814ab81-ce9a-4faf-9be6-f0a90c416286 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:17.282654 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0814ab81-ce9a-4faf-9be6-f0a90c416286 None admin] RBAC: Adding query filter params (role.id=e22e4d2b3d06480bbcfa4d2ee3387c90, scope.project.id=d6cc740b2ba549809a3b7f863d43517c, user.id=fa2b48749a564275b6ac6bed107a81a5) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:17.283210 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0814ab81-ce9a-4faf-9be6-f0a90c416286 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:17.284847 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0814ab81-ce9a-4faf-9be6-f0a90c416286 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:17.291962 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 104/205] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:34:17 2022] GET /identity/v3/role_assignments?user.id=fa2b48749a564275b6ac6bed107a81a5&scope.project.id=d6cc740b2ba549809a3b7f863d43517c&role.id=e22e4d2b3d06480bbcfa4d2ee3387c90 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:17.329312 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 102/206] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:34:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 647 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:34:17.335821 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0814ab81-ce9a-4faf-9be6-f0a90c416286 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:17.463306 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:17.486697 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0814ab81-ce9a-4faf-9be6-f0a90c416286 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:17.531419 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da124a78-ea2d-4d00-9147-7870c16c828a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:17.532396 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da124a78-ea2d-4d00-9147-7870c16c828a None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:17.532878 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da124a78-ea2d-4d00-9147-7870c16c828a None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:17.533381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da124a78-ea2d-4d00-9147-7870c16c828a None admin] PATH_INFO: `/v3/projects/2694f62f168849c0a5cd44bffc841cef` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:17.534770 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da124a78-ea2d-4d00-9147-7870c16c828a None admin] RBAC: Authorizing `identity:get_project(project_id=2694f62f168849c0a5cd44bffc841cef)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:17.536588 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da124a78-ea2d-4d00-9147-7870c16c828a None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:17.538801 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 105/207] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:17 2022] GET /identity/v3/projects/2694f62f168849c0a5cd44bffc841cef => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:17.544947 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0305b71-4bc3-4644-93bc-278d8a0eb713 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:17.569817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:17.672440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0305b71-4bc3-4644-93bc-278d8a0eb713 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:17.712014 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62346a7-e84d-4165-bbff-606400e32837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:17.712961 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62346a7-e84d-4165-bbff-606400e32837 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:17.713476 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62346a7-e84d-4165-bbff-606400e32837 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:17.713957 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62346a7-e84d-4165-bbff-606400e32837 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:17.715150 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62346a7-e84d-4165-bbff-606400e32837 None admin] RBAC: Adding query filter params (role.id=05a9201bdeee4f2ab18c648d5e6cb250, scope.project.id=2694f62f168849c0a5cd44bffc841cef, user.id=9ddd8e696aa24e88917377eb6e3199da) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:17.715663 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62346a7-e84d-4165-bbff-606400e32837 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:17.718874 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62346a7-e84d-4165-bbff-606400e32837 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:17.726743 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 103/208] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:34:17 2022] GET /identity/v3/role_assignments?user.id=9ddd8e696aa24e88917377eb6e3199da&scope.project.id=2694f62f168849c0a5cd44bffc841cef&role.id=05a9201bdeee4f2ab18c648d5e6cb250 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:18.674574 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9281742-bd48-4a57-a528-4c0a538a1d82 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:18.675106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9281742-bd48-4a57-a528-4c0a538a1d82 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:18.675106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9281742-bd48-4a57-a528-4c0a538a1d82 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:18.677174 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 106/209] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:18.682517 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addc715-1590-429a-8d25-bcdfa721c53d None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:18.682936 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addc715-1590-429a-8d25-bcdfa721c53d None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:18.683009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addc715-1590-429a-8d25-bcdfa721c53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:18.701868 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-6addc715-1590-429a-8d25-bcdfa721c53d None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:18.703087 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-6addc715-1590-429a-8d25-bcdfa721c53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:18.710249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622308ca-07e8-4a43-9709-9fb377c40c74 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:18.710535 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622308ca-07e8-4a43-9709-9fb377c40c74 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:18.710855 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622308ca-07e8-4a43-9709-9fb377c40c74 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:18.712540 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 107/210] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:18.718143 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ecf55-4560-475e-9cbb-d670601177e6 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:18.718431 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ecf55-4560-475e-9cbb-d670601177e6 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:18.718707 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ecf55-4560-475e-9cbb-d670601177e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:18.740931 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-5c1ecf55-4560-475e-9cbb-d670601177e6 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:18.742153 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-5c1ecf55-4560-475e-9cbb-d670601177e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:18.779122 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 104/211] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:18.803991 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6addc715-1590-429a-8d25-bcdfa721c53d None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:18.806483 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:18.816619 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 108/212] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:34:18.841211 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ecf55-4560-475e-9cbb-d670601177e6 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:18.923448 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6addc715-1590-429a-8d25-bcdfa721c53d None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:18.939434 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:18.961005 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ecf55-4560-475e-9cbb-d670601177e6 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:18.968125 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9811e636-1cd3-43c4-b4e9-3c145dd97e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:18.968776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9811e636-1cd3-43c4-b4e9-3c145dd97e47 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:18.969103 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9811e636-1cd3-43c4-b4e9-3c145dd97e47 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:18.969408 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9811e636-1cd3-43c4-b4e9-3c145dd97e47 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:18.984578 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9811e636-1cd3-43c4-b4e9-3c145dd97e47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:18.991907 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-9811e636-1cd3-43c4-b4e9-3c145dd97e47 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:18.991907 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:18.991907 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:18.991907 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:18.991907 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:18.991907 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:18.991907 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:18.991907 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:18.991907 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:18.991907 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:18.991907 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:18.991907 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:18.991907 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:18.991907 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:18.991907 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:18.991907 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:18.992799 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:18.992799 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:18.992799 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:18.992799 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:18.992799 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:18.992799 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:18.992799 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:18.992799 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:18.992799 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:18.992799 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:18.992799 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:18.992799 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:18.992799 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:18.992799 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:18.992799 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:18.992799 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:18.993599 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:18.993599 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:18.993599 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:18.993599 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:18.993599 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:18.993599 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:18.993599 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:18.993599 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:18.993599 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:18.993599 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:18.993599 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:18.993599 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:18.993599 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 105/213] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:34:18.999044 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9811e636-1cd3-43c4-b4e9-3c145dd97e47 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:19.003401 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22ff1d-adba-4adc-85c9-18524be7c626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:19.004159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22ff1d-adba-4adc-85c9-18524be7c626 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:19.004617 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22ff1d-adba-4adc-85c9-18524be7c626 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:19.004992 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22ff1d-adba-4adc-85c9-18524be7c626 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:19.006291 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b22ff1d-adba-4adc-85c9-18524be7c626 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:19.009032 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b22ff1d-adba-4adc-85c9-18524be7c626 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:19.011101 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 109/214] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:34:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:34:19.016680 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22ff1d-adba-4adc-85c9-18524be7c626 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:19.071240 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:19.117975 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9811e636-1cd3-43c4-b4e9-3c145dd97e47 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:19.137632 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22ff1d-adba-4adc-85c9-18524be7c626 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:19.160562 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1741c1-14ee-4cf8-bddb-f09ea384f094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:19.161469 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1741c1-14ee-4cf8-bddb-f09ea384f094 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:19.161747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1741c1-14ee-4cf8-bddb-f09ea384f094 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:19.162346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1741c1-14ee-4cf8-bddb-f09ea384f094 None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:19.163291 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1741c1-14ee-4cf8-bddb-f09ea384f094 None admin] RBAC: Adding query filter params (name=member) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:19.163597 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1741c1-14ee-4cf8-bddb-f09ea384f094 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:19.165500 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1741c1-14ee-4cf8-bddb-f09ea384f094 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:19.172184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8b740-ab0d-48f4-867f-c89cb4fff8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:19.172799 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8b740-ab0d-48f4-867f-c89cb4fff8c9 None admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:19.173076 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8b740-ab0d-48f4-867f-c89cb4fff8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:19.173438 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8b740-ab0d-48f4-867f-c89cb4fff8c9 None admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:19.174012 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc8b740-ab0d-48f4-867f-c89cb4fff8c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:19.175446 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc8b740-ab0d-48f4-867f-c89cb4fff8c9 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:19.181079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 106/215] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 10 22:34:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:34:19.181079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a50ac90-a830-4585-b7b5-5a9ba691e0e2 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:19.181079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a50ac90-a830-4585-b7b5-5a9ba691e0e2 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:19.181079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a50ac90-a830-4585-b7b5-5a9ba691e0e2 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:19.187832 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 107/216] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:19.187832 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a50ac90-a830-4585-b7b5-5a9ba691e0e2 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:19.210142 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 110/217] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 10 22:34:19 2022] POST /identity/v3/users => generated 297 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:34:19.211688 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9d14b-958e-4b7f-9e7a-4f39c2de615a None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:19.212000 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9d14b-958e-4b7f-9e7a-4f39c2de615a None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:19.212276 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9d14b-958e-4b7f-9e7a-4f39c2de615a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:19.233014 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:19.250457 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-7ab9d14b-958e-4b7f-9e7a-4f39c2de615a None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:19.252939 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-7ab9d14b-958e-4b7f-9e7a-4f39c2de615a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:19.354969 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a50ac90-a830-4585-b7b5-5a9ba691e0e2 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:19.372444 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 111/218] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:19.396498 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab9d14b-958e-4b7f-9e7a-4f39c2de615a None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:19.400964 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:19.413510 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd18098-5346-4535-9a00-4c8a0bfb3cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:19.415010 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd18098-5346-4535-9a00-4c8a0bfb3cd0 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:19.415010 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd18098-5346-4535-9a00-4c8a0bfb3cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:19.415010 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd18098-5346-4535-9a00-4c8a0bfb3cd0 None admin] PATH_INFO: `/v3/users/c6e0b984433648909afcb604f5f12ad8` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:19.435702 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd18098-5346-4535-9a00-4c8a0bfb3cd0 None admin] RBAC: Authorizing `identity:get_user(user_id=c6e0b984433648909afcb604f5f12ad8)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:19.437914 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd18098-5346-4535-9a00-4c8a0bfb3cd0 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:19.439986 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 108/219] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:34:19 2022] GET /identity/v3/users/c6e0b984433648909afcb604f5f12ad8 => generated 297 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:19.445614 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd18098-5346-4535-9a00-4c8a0bfb3cd0 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:19.540712 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab9d14b-958e-4b7f-9e7a-4f39c2de615a None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:19.591477 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd18098-5346-4535-9a00-4c8a0bfb3cd0 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:19.597626 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0bf1e8-d84b-42ec-a416-e3b6d63c98e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:19.598829 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0bf1e8-d84b-42ec-a416-e3b6d63c98e6 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:19.599254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0bf1e8-d84b-42ec-a416-e3b6d63c98e6 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:19.599527 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0bf1e8-d84b-42ec-a416-e3b6d63c98e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:19.602651 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:19.612835 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0bf1e8-d84b-42ec-a416-e3b6d63c98e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:19.618666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-3e0bf1e8-d84b-42ec-a416-e3b6d63c98e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:34:19.618666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:19.618666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:19.618666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:19.618666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:19.618666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:19.618666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:19.618666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:19.618666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:19.618666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:19.618666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:19.618666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:19.618666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:19.618666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:19.618666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:19.618666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:19.619553 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:19.619553 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:19.619553 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:19.619553 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:19.619553 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:19.619553 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:19.619553 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:19.619553 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:19.619553 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:19.619553 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:19.619553 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:19.619553 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:19.619553 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:19.619553 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:19.619553 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:19.619553 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:19.620328 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:19.620328 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:19.620328 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:19.620328 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:19.620328 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:19.620328 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:19.620328 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:19.620328 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:19.620328 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:19.620328 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:19.620328 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:34:19.620328 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:19.620328 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 112/220] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:34:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:34:19.630510 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0bf1e8-d84b-42ec-a416-e3b6d63c98e6 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:19.652426 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e5c6f-4798-4a84-aba5-83b09cd5e24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:19.653069 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e5c6f-4798-4a84-aba5-83b09cd5e24d None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:19.653363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e5c6f-4798-4a84-aba5-83b09cd5e24d None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:19.653645 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e5c6f-4798-4a84-aba5-83b09cd5e24d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:19.719352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:19.742221 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72e5c6f-4798-4a84-aba5-83b09cd5e24d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:19.743968 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72e5c6f-4798-4a84-aba5-83b09cd5e24d None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:19.752310 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0bf1e8-d84b-42ec-a416-e3b6d63c98e6 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:19.791348 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5029ff-0e40-401a-bba1-3ad35c3ee6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:19.792258 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5029ff-0e40-401a-bba1-3ad35c3ee6d6 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:19.792712 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5029ff-0e40-401a-bba1-3ad35c3ee6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:19.793157 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5029ff-0e40-401a-bba1-3ad35c3ee6d6 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:19.794113 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5029ff-0e40-401a-bba1-3ad35c3ee6d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:19.794704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5029ff-0e40-401a-bba1-3ad35c3ee6d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:19.796248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5029ff-0e40-401a-bba1-3ad35c3ee6d6 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:19.806163 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 113/221] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:19.807805 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ae104-6312-4f6b-9cc1-9d84d0959cbc None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:19.808378 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ae104-6312-4f6b-9cc1-9d84d0959cbc None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:19.808828 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ae104-6312-4f6b-9cc1-9d84d0959cbc None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:19.810651 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 114/222] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:19.811745 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ae104-6312-4f6b-9cc1-9d84d0959cbc None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:19.855581 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 109/223] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:34:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 22:34:19.860318 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b70800-8bb7-4d6b-9075-5eddd5eeafa8 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:19.860318 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b70800-8bb7-4d6b-9075-5eddd5eeafa8 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:19.860318 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b70800-8bb7-4d6b-9075-5eddd5eeafa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:19.877488 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-a1b70800-8bb7-4d6b-9075-5eddd5eeafa8 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:19.878873 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-a1b70800-8bb7-4d6b-9075-5eddd5eeafa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:19.883508 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:19.931487 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ae104-6312-4f6b-9cc1-9d84d0959cbc None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:19.959509 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 110/224] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:19.960585 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b70800-8bb7-4d6b-9075-5eddd5eeafa8 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:19.971556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e70822-04b1-4818-aa8d-874d900e068d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:19.973096 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e70822-04b1-4818-aa8d-874d900e068d None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:19.973633 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e70822-04b1-4818-aa8d-874d900e068d None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:19.974228 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e70822-04b1-4818-aa8d-874d900e068d None admin] PATH_INFO: `/v3/users/fa2b48749a564275b6ac6bed107a81a5` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:19.975604 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e70822-04b1-4818-aa8d-874d900e068d None admin] RBAC: Authorizing `identity:get_user(user_id=fa2b48749a564275b6ac6bed107a81a5)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:19.976986 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:19.978620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e70822-04b1-4818-aa8d-874d900e068d None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:19.981814 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 115/225] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:34:19 2022] GET /identity/v3/users/fa2b48749a564275b6ac6bed107a81a5 => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:19.985392 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e70822-04b1-4818-aa8d-874d900e068d None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:20.067713 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b70800-8bb7-4d6b-9075-5eddd5eeafa8 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:20.092274 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e70822-04b1-4818-aa8d-874d900e068d None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:20.102469 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94d356-2bab-4148-9719-ba0a210aa728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:20.103111 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94d356-2bab-4148-9719-ba0a210aa728 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:20.103622 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94d356-2bab-4148-9719-ba0a210aa728 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:20.103622 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94d356-2bab-4148-9719-ba0a210aa728 None admin] PATH_INFO: `/v3/projects/d6cc740b2ba549809a3b7f863d43517c` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:20.104864 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f94d356-2bab-4148-9719-ba0a210aa728 None admin] RBAC: Authorizing `identity:get_project(project_id=d6cc740b2ba549809a3b7f863d43517c)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:20.106309 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f94d356-2bab-4148-9719-ba0a210aa728 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:20.107978 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 111/226] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:19 2022] GET /identity/v3/projects/d6cc740b2ba549809a3b7f863d43517c => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:20.113466 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94d356-2bab-4148-9719-ba0a210aa728 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:20.133653 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:20.144133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e8343-2db7-45c4-8b0a-79a04758fd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:20.145356 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e8343-2db7-45c4-8b0a-79a04758fd7f None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:20.146134 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e8343-2db7-45c4-8b0a-79a04758fd7f None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:20.146987 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e8343-2db7-45c4-8b0a-79a04758fd7f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:20.162256 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6e8343-2db7-45c4-8b0a-79a04758fd7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:20.170897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-bc6e8343-2db7-45c4-8b0a-79a04758fd7f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:34:20.170897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:20.170897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:20.170897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:20.170897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:20.170897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:20.170897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:20.170897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:20.170897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:20.170897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:20.170897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:20.170897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:20.170897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:20.170897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:20.170897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:20.170897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:20.171794 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:20.171794 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:20.171794 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:20.171794 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:20.171794 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:20.171794 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:20.171794 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:20.171794 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:20.171794 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:20.171794 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:20.171794 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:20.171794 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:20.171794 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:20.171794 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:20.171794 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:20.171794 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:20.172570 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:20.172570 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:20.172570 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:20.172570 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:20.172570 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:20.172570 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:20.172570 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:20.172570 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:20.172570 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:20.172570 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:20.172570 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:34:20.172570 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:20.176112 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 116/227] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 10 22:34:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:20.177090 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e8343-2db7-45c4-8b0a-79a04758fd7f None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:20.216864 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:20.244972 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94d356-2bab-4148-9719-ba0a210aa728 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:20.287088 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86d229-2788-4edf-9a43-a5490775ce60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:20.287760 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86d229-2788-4edf-9a43-a5490775ce60 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:20.288066 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86d229-2788-4edf-9a43-a5490775ce60 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:20.288522 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86d229-2788-4edf-9a43-a5490775ce60 None admin] PATH_INFO: `/v3/projects/d6cc740b2ba549809a3b7f863d43517c` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:20.289479 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b86d229-2788-4edf-9a43-a5490775ce60 None admin] RBAC: Authorizing `identity:get_project(project_id=d6cc740b2ba549809a3b7f863d43517c)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:20.290953 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b86d229-2788-4edf-9a43-a5490775ce60 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:20.295319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e8343-2db7-45c4-8b0a-79a04758fd7f None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:20.297658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 112/228] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:20 2022] GET /identity/v3/projects/d6cc740b2ba549809a3b7f863d43517c => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:20.298741 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86d229-2788-4edf-9a43-a5490775ce60 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:20.333617 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c80c6-3018-4d20-9dd5-daa4e19fecb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:20.334825 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c80c6-3018-4d20-9dd5-daa4e19fecb1 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:20.336151 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c80c6-3018-4d20-9dd5-daa4e19fecb1 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:20.337205 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c80c6-3018-4d20-9dd5-daa4e19fecb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:20.338290 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c80c6-3018-4d20-9dd5-daa4e19fecb1 None admin] RBAC: Adding query filter params (role.id=1085473c6e534f87812b05c7014f77c2, scope.project.id=d6cc740b2ba549809a3b7f863d43517c, user.id=c6e0b984433648909afcb604f5f12ad8) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:20.338876 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c80c6-3018-4d20-9dd5-daa4e19fecb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:20.340481 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c80c6-3018-4d20-9dd5-daa4e19fecb1 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:20.348161 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 117/229] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:34:20 2022] GET /identity/v3/role_assignments?user.id=c6e0b984433648909afcb604f5f12ad8&scope.project.id=d6cc740b2ba549809a3b7f863d43517c&role.id=1085473c6e534f87812b05c7014f77c2 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:20.349392 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c80c6-3018-4d20-9dd5-daa4e19fecb1 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:20.411835 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:20.474916 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c80c6-3018-4d20-9dd5-daa4e19fecb1 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:20.506020 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86d229-2788-4edf-9a43-a5490775ce60 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:20.517253 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d485964f-2e14-4cba-bc69-05f06eb12e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:20.518981 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485964f-2e14-4cba-bc69-05f06eb12e61 None admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:20.519449 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485964f-2e14-4cba-bc69-05f06eb12e61 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:20.520555 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485964f-2e14-4cba-bc69-05f06eb12e61 None admin] PATH_INFO: `/v3/projects/d6cc740b2ba549809a3b7f863d43517c/users/fa2b48749a564275b6ac6bed107a81a5/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:20.522782 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d485964f-2e14-4cba-bc69-05f06eb12e61 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6cc740b2ba549809a3b7f863d43517c, user_id=fa2b48749a564275b6ac6bed107a81a5, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:20.524448 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d485964f-2e14-4cba-bc69-05f06eb12e61 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:20.548051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 118/230] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 10 22:34:20 2022] PUT /identity/v3/projects/d6cc740b2ba549809a3b7f863d43517c/users/fa2b48749a564275b6ac6bed107a81a5/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:34:20.565765 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3cf4e-cba9-4454-9235-f12b7984a3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:20.566399 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3cf4e-cba9-4454-9235-f12b7984a3cf None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:20.566676 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3cf4e-cba9-4454-9235-f12b7984a3cf None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:20.567023 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3cf4e-cba9-4454-9235-f12b7984a3cf None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:20.567672 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b3cf4e-cba9-4454-9235-f12b7984a3cf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:20.567953 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b3cf4e-cba9-4454-9235-f12b7984a3cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:20.569519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b3cf4e-cba9-4454-9235-f12b7984a3cf None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:20.580055 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 113/231] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:34:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:20.585988 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d485964f-2e14-4cba-bc69-05f06eb12e61 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:20.718102 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d485964f-2e14-4cba-bc69-05f06eb12e61 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:20.729384 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:20.756573 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f3d28-7318-459e-a883-e051d1d20edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:20.757317 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f3d28-7318-459e-a883-e051d1d20edc None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:20.757625 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f3d28-7318-459e-a883-e051d1d20edc None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:20.757921 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f3d28-7318-459e-a883-e051d1d20edc None admin] PATH_INFO: `/v3/users/fa2b48749a564275b6ac6bed107a81a5` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:20.759018 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12f3d28-7318-459e-a883-e051d1d20edc None admin] RBAC: Authorizing `identity:get_user(user_id=fa2b48749a564275b6ac6bed107a81a5)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:20.761046 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12f3d28-7318-459e-a883-e051d1d20edc None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:20.763028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 119/232] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:34:20 2022] GET /identity/v3/users/fa2b48749a564275b6ac6bed107a81a5 => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:20.768534 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3cf4e-cba9-4454-9235-f12b7984a3cf None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:20.782267 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:20.877704 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3cf4e-cba9-4454-9235-f12b7984a3cf None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:20.912565 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f8755-e0d5-4d6d-81f5-60894885af77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:20.913192 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f8755-e0d5-4d6d-81f5-60894885af77 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:20.913508 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f8755-e0d5-4d6d-81f5-60894885af77 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:20.913776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f8755-e0d5-4d6d-81f5-60894885af77 None admin] PATH_INFO: `/v3/projects/d6cc740b2ba549809a3b7f863d43517c` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:20.914891 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f8755-e0d5-4d6d-81f5-60894885af77 None admin] RBAC: Authorizing `identity:get_project(project_id=d6cc740b2ba549809a3b7f863d43517c)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:20.916294 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f8755-e0d5-4d6d-81f5-60894885af77 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:20.918389 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 114/233] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:20 2022] GET /identity/v3/projects/d6cc740b2ba549809a3b7f863d43517c => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:20.923961 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f3d28-7318-459e-a883-e051d1d20edc None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:21.048153 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f3d28-7318-459e-a883-e051d1d20edc None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:21.085803 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5871b2-7c82-4a88-adaf-7934c7f33df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:21.086472 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5871b2-7c82-4a88-adaf-7934c7f33df9 None admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:21.086730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5871b2-7c82-4a88-adaf-7934c7f33df9 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:21.087001 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5871b2-7c82-4a88-adaf-7934c7f33df9 None admin] PATH_INFO: `/v3/projects/d6cc740b2ba549809a3b7f863d43517c/users/fa2b48749a564275b6ac6bed107a81a5/roles/e22e4d2b3d06480bbcfa4d2ee3387c90` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:21.089148 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5871b2-7c82-4a88-adaf-7934c7f33df9 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6cc740b2ba549809a3b7f863d43517c, user_id=fa2b48749a564275b6ac6bed107a81a5, role_id=e22e4d2b3d06480bbcfa4d2ee3387c90)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:21.090654 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5871b2-7c82-4a88-adaf-7934c7f33df9 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:21.106829 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 120/234] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 10 22:34:20 2022] PUT /identity/v3/projects/d6cc740b2ba549809a3b7f863d43517c/users/fa2b48749a564275b6ac6bed107a81a5/roles/e22e4d2b3d06480bbcfa4d2ee3387c90 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:34:21.739574 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c421d83-afc9-4ac4-8a0c-da62692ad93c None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:21.740225 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c421d83-afc9-4ac4-8a0c-da62692ad93c None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:21.740707 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c421d83-afc9-4ac4-8a0c-da62692ad93c None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:21.742483 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 115/235] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:21.748217 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db58bf7-fb72-4f35-a735-f86594eb7697 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:21.748767 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db58bf7-fb72-4f35-a735-f86594eb7697 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:21.749247 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db58bf7-fb72-4f35-a735-f86594eb7697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:21.778389 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-4db58bf7-fb72-4f35-a735-f86594eb7697 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:21.780354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-4db58bf7-fb72-4f35-a735-f86594eb7697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:21.815665 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:21.854435 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0770b-1a3b-4d8c-b204-f0f96ffbd3f2 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:21.854963 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0770b-1a3b-4d8c-b204-f0f96ffbd3f2 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:21.854963 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0770b-1a3b-4d8c-b204-f0f96ffbd3f2 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:21.856384 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 116/236] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:21.858638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 121/237] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:21.866748 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc70dd-9ce5-47a6-8c8e-ac9eb2a75dc0 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:21.867021 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc70dd-9ce5-47a6-8c8e-ac9eb2a75dc0 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:21.867287 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc70dd-9ce5-47a6-8c8e-ac9eb2a75dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:21.886219 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db58bf7-fb72-4f35-a735-f86594eb7697 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:21.902273 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-4acc70dd-9ce5-47a6-8c8e-ac9eb2a75dc0 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:21.903452 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-4acc70dd-9ce5-47a6-8c8e-ac9eb2a75dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:22.000297 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db58bf7-fb72-4f35-a735-f86594eb7697 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:22.039559 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563bce60-117d-4c07-8348-51f81c11c527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:22.040673 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563bce60-117d-4c07-8348-51f81c11c527 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:22.041269 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563bce60-117d-4c07-8348-51f81c11c527 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:22.041766 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563bce60-117d-4c07-8348-51f81c11c527 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:22.043054 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563bce60-117d-4c07-8348-51f81c11c527 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:22.044744 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563bce60-117d-4c07-8348-51f81c11c527 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:22.046532 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 122/238] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:34:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:34:22.053081 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563bce60-117d-4c07-8348-51f81c11c527 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:22.060155 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 117/239] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:22.084285 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc70dd-9ce5-47a6-8c8e-ac9eb2a75dc0 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:22.103213 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:22.108743 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:22.180770 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563bce60-117d-4c07-8348-51f81c11c527 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:22.221305 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29cdb8-ca29-491c-8a16-0e462a0dfb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:22.222177 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29cdb8-ca29-491c-8a16-0e462a0dfb3b None admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:22.222586 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29cdb8-ca29-491c-8a16-0e462a0dfb3b None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:22.223251 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29cdb8-ca29-491c-8a16-0e462a0dfb3b None admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:22.225144 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e29cdb8-ca29-491c-8a16-0e462a0dfb3b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:22.226725 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e29cdb8-ca29-491c-8a16-0e462a0dfb3b None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:22.274635 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 123/240] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 10 22:34:22 2022] POST /identity/v3/users => generated 293 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:34:22.303214 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc70dd-9ce5-47a6-8c8e-ac9eb2a75dc0 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:22.355801 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bece34-1660-42e0-a09c-063894ef2861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:22.356678 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bece34-1660-42e0-a09c-063894ef2861 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:22.357100 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bece34-1660-42e0-a09c-063894ef2861 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:22.357538 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bece34-1660-42e0-a09c-063894ef2861 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:22.374385 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bece34-1660-42e0-a09c-063894ef2861 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:22.380787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-77bece34-1660-42e0-a09c-063894ef2861 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:34:22.380787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:22.380787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:22.380787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:22.380787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:22.380787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:22.380787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:22.380787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:22.380787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:22.380787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:22.380787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:22.380787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:22.380787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:22.380787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:22.380787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:22.380787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:22.381850 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:22.381850 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:22.381850 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:22.381850 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:22.381850 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:22.381850 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:22.381850 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:22.381850 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:22.381850 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:22.381850 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:22.381850 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:22.381850 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:22.381850 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:22.381850 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:22.381850 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:22.381850 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:22.382763 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:22.382763 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:22.382763 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:22.382763 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:22.382763 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:22.382763 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:22.382763 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:22.382763 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:22.382763 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:22.382763 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:22.382763 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:34:22.382763 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:22.385139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 118/241] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:22.391362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29cdb8-ca29-491c-8a16-0e462a0dfb3b None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:22.547244 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:22.561354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29cdb8-ca29-491c-8a16-0e462a0dfb3b None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:22.605432 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98116856-22fd-495f-8253-e0332444af36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:22.606050 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98116856-22fd-495f-8253-e0332444af36 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:22.606331 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98116856-22fd-495f-8253-e0332444af36 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:22.606601 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98116856-22fd-495f-8253-e0332444af36 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:22.607276 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98116856-22fd-495f-8253-e0332444af36 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:22.607576 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98116856-22fd-495f-8253-e0332444af36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:22.608973 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98116856-22fd-495f-8253-e0332444af36 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:22.622214 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 124/242] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 10 22:34:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:22.627573 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bece34-1660-42e0-a09c-063894ef2861 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:22.708312 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:22.749709 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bece34-1660-42e0-a09c-063894ef2861 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:22.792460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd8fd5-1b0a-4108-9f03-bae72db8d868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:22.793105 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd8fd5-1b0a-4108-9f03-bae72db8d868 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:22.793415 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd8fd5-1b0a-4108-9f03-bae72db8d868 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:22.793699 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd8fd5-1b0a-4108-9f03-bae72db8d868 None admin] PATH_INFO: `/v3/users/192102ee45ad40728357b262abd23a82` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:22.813021 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd8fd5-1b0a-4108-9f03-bae72db8d868 None admin] RBAC: Authorizing `identity:get_user(user_id=192102ee45ad40728357b262abd23a82)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:22.814546 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd8fd5-1b0a-4108-9f03-bae72db8d868 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:22.816521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 119/243] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:34:22 2022] GET /identity/v3/users/192102ee45ad40728357b262abd23a82 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:34:22.821957 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98116856-22fd-495f-8253-e0332444af36 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:22.952401 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98116856-22fd-495f-8253-e0332444af36 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:22.993885 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc4129-dd88-47e4-8479-e169f0ab86ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:22.994556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc4129-dd88-47e4-8479-e169f0ab86ba None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:22.994915 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc4129-dd88-47e4-8479-e169f0ab86ba None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:22.995295 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc4129-dd88-47e4-8479-e169f0ab86ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:23.014814 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b123af-e114-4cec-be7c-fe2d9e0ac29c None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:23.015308 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b123af-e114-4cec-be7c-fe2d9e0ac29c None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:23.015379 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b123af-e114-4cec-be7c-fe2d9e0ac29c None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:23.018277 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 120/244] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:23.023519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b620b9e4-dcf7-40a8-922f-9f9b4f7386eb None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:23.023805 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b620b9e4-dcf7-40a8-922f-9f9b4f7386eb None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:23.024081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b620b9e4-dcf7-40a8-922f-9f9b4f7386eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:23.028041 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:23.055427 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-b620b9e4-dcf7-40a8-922f-9f9b4f7386eb None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:23.057248 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-b620b9e4-dcf7-40a8-922f-9f9b4f7386eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:23.082488 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cc4129-dd88-47e4-8479-e169f0ab86ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:23.083878 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cc4129-dd88-47e4-8479-e169f0ab86ba None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:23.149855 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 121/245] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:23.152003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145285b4-92b9-44a3-bf7b-901833a8907a None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:23.152288 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145285b4-92b9-44a3-bf7b-901833a8907a None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:23.152691 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145285b4-92b9-44a3-bf7b-901833a8907a None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:23.154765 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 122/246] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:23.160111 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d8906-8341-4ff1-88da-6c9d695c57e9 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:23.160393 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d8906-8341-4ff1-88da-6c9d695c57e9 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:23.160666 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d8906-8341-4ff1-88da-6c9d695c57e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:23.181761 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-292d8906-8341-4ff1-88da-6c9d695c57e9 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:23.182942 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-292d8906-8341-4ff1-88da-6c9d695c57e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:23.198642 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 125/247] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:34:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:34:23.199502 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc4129-dd88-47e4-8479-e169f0ab86ba None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:23.255349 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:23.257095 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:23.283795 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 123/248] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:34:23.284643 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d8906-8341-4ff1-88da-6c9d695c57e9 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:23.339912 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc4129-dd88-47e4-8479-e169f0ab86ba None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:23.405120 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548f247-2e1b-439e-a014-84a0b29e349c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:23.405798 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548f247-2e1b-439e-a014-84a0b29e349c None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:23.406072 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548f247-2e1b-439e-a014-84a0b29e349c None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:23.406340 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548f247-2e1b-439e-a014-84a0b29e349c None admin] PATH_INFO: `/v3/roles/member` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:23.436292 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b548f247-2e1b-439e-a014-84a0b29e349c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:23.443248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-b548f247-2e1b-439e-a014-84a0b29e349c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:23.443248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:23.443248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:23.443248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:23.443248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:23.443248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:23.443248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:23.443248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:23.443248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:23.443248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:23.443248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:23.443248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:23.443248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:23.443248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:23.443248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:23.443248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:23.444118 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:23.444118 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:23.444118 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:23.444118 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:23.444118 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:23.444118 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:23.444118 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:23.444118 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:23.444118 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:23.444118 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:23.444118 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:23.444118 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:23.444118 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:23.444118 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:23.444118 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:23.444118 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:23.444879 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:23.444879 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:23.444879 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:23.444879 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:23.444879 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:23.444879 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:23.444879 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:23.444879 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:23.444879 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:23.444879 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:23.444879 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:23.444879 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:23.447819 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 126/249] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:23.448980 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548f247-2e1b-439e-a014-84a0b29e349c None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:23.547651 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d8906-8341-4ff1-88da-6c9d695c57e9 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:23.590731 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548f247-2e1b-439e-a014-84a0b29e349c None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:23.605569 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff94d4-d31d-4977-8730-a9455c3ac59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:23.606924 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff94d4-d31d-4977-8730-a9455c3ac59f None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:23.607205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff94d4-d31d-4977-8730-a9455c3ac59f None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:23.614078 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:23.614897 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff94d4-d31d-4977-8730-a9455c3ac59f None admin] PATH_INFO: `/v3/projects/d6cc740b2ba549809a3b7f863d43517c` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:23.616196 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eff94d4-d31d-4977-8730-a9455c3ac59f None admin] RBAC: Authorizing `identity:get_project(project_id=d6cc740b2ba549809a3b7f863d43517c)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:23.618669 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eff94d4-d31d-4977-8730-a9455c3ac59f None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:23.620673 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 124/250] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:23 2022] GET /identity/v3/projects/d6cc740b2ba549809a3b7f863d43517c => generated 302 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:23.622138 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff94d4-d31d-4977-8730-a9455c3ac59f None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:23.670833 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aeb63e-d775-4307-9d65-60895d5d0de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:23.671462 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aeb63e-d775-4307-9d65-60895d5d0de3 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:23.671734 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aeb63e-d775-4307-9d65-60895d5d0de3 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:23.671999 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aeb63e-d775-4307-9d65-60895d5d0de3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:23.694238 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aeb63e-d775-4307-9d65-60895d5d0de3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:23.700283 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:23.702281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-c4aeb63e-d775-4307-9d65-60895d5d0de3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:34:23.702281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:23.702281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:23.702281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:23.702281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:23.702281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:23.702281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:23.702281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:23.702281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:23.702281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:23.702281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:23.702281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:23.702281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:23.702281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:23.702281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:23.702281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:23.703225 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:23.703225 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:23.703225 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:23.703225 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:23.703225 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:23.703225 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:23.703225 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:23.703225 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:23.703225 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:23.703225 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:23.703225 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:23.703225 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:23.703225 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:23.703225 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:23.703225 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:23.703225 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:23.704054 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:23.704054 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:23.704054 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:23.704054 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:23.704054 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:23.704054 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:23.704054 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:23.704054 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:23.704054 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:23.704054 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:23.704054 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:34:23.704054 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:23.710314 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 127/251] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:34:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:23.712878 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aeb63e-d775-4307-9d65-60895d5d0de3 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:23.776256 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff94d4-d31d-4977-8730-a9455c3ac59f None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:23.813256 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665756a-e0e6-47c0-a2d2-e01c5d9a6ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:23.813740 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665756a-e0e6-47c0-a2d2-e01c5d9a6ecd None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:23.814013 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665756a-e0e6-47c0-a2d2-e01c5d9a6ecd None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:23.814281 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665756a-e0e6-47c0-a2d2-e01c5d9a6ecd None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:23.814931 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4665756a-e0e6-47c0-a2d2-e01c5d9a6ecd None admin] RBAC: Adding query filter params (name=member) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:23.815226 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4665756a-e0e6-47c0-a2d2-e01c5d9a6ecd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:23.816594 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4665756a-e0e6-47c0-a2d2-e01c5d9a6ecd None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:23.826121 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 125/252] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 10 22:34:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:23.827029 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665756a-e0e6-47c0-a2d2-e01c5d9a6ecd None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:23.847431 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aeb63e-d775-4307-9d65-60895d5d0de3 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:23.889136 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae8ddc-bb66-4e33-b0be-1aadd03ab888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:23.890157 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae8ddc-bb66-4e33-b0be-1aadd03ab888 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:23.890690 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae8ddc-bb66-4e33-b0be-1aadd03ab888 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:23.891153 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae8ddc-bb66-4e33-b0be-1aadd03ab888 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:23.892074 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae8ddc-bb66-4e33-b0be-1aadd03ab888 None admin] RBAC: Adding query filter params (role.id=05a9201bdeee4f2ab18c648d5e6cb250, scope.project.id=d6cc740b2ba549809a3b7f863d43517c, user.id=192102ee45ad40728357b262abd23a82) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:23.892572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae8ddc-bb66-4e33-b0be-1aadd03ab888 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:23.894251 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae8ddc-bb66-4e33-b0be-1aadd03ab888 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:23.901196 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 128/253] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:34:23 2022] GET /identity/v3/role_assignments?user.id=192102ee45ad40728357b262abd23a82&scope.project.id=d6cc740b2ba549809a3b7f863d43517c&role.id=05a9201bdeee4f2ab18c648d5e6cb250 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:23.902881 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beedd222-8fb5-49b9-931f-8e50c632947d None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:23.903360 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beedd222-8fb5-49b9-931f-8e50c632947d None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:23.903828 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beedd222-8fb5-49b9-931f-8e50c632947d None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:23.905471 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 129/254] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:23.912894 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beedd222-8fb5-49b9-931f-8e50c632947d None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:23.949725 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665756a-e0e6-47c0-a2d2-e01c5d9a6ecd None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:23.962273 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:23.993341 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce5e008-86af-441c-a62f-e65353134818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:23.994199 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5e008-86af-441c-a62f-e65353134818 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:23.994648 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5e008-86af-441c-a62f-e65353134818 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:23.995064 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5e008-86af-441c-a62f-e65353134818 None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:23.995988 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce5e008-86af-441c-a62f-e65353134818 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:23.996431 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce5e008-86af-441c-a62f-e65353134818 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:23.998168 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce5e008-86af-441c-a62f-e65353134818 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:24.005648 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:24.010559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 126/255] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:24.016872 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cceba51-069a-4b2b-942a-872422065db9 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:24.017504 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cceba51-069a-4b2b-942a-872422065db9 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:24.017504 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cceba51-069a-4b2b-942a-872422065db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:24.056049 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-4cceba51-069a-4b2b-942a-872422065db9 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:24.057321 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-4cceba51-069a-4b2b-942a-872422065db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:24.099262 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beedd222-8fb5-49b9-931f-8e50c632947d None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:24.148989 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 127/256] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:24.149901 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cceba51-069a-4b2b-942a-872422065db9 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:24.162194 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d8df89-fa1d-43f3-a297-902c484facf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:24.162842 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8df89-fa1d-43f3-a297-902c484facf2 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:24.163222 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8df89-fa1d-43f3-a297-902c484facf2 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:24.163496 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8df89-fa1d-43f3-a297-902c484facf2 None admin] PATH_INFO: `/v3/users/c6e0b984433648909afcb604f5f12ad8` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:24.166738 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d8df89-fa1d-43f3-a297-902c484facf2 None admin] RBAC: Authorizing `identity:get_user(user_id=c6e0b984433648909afcb604f5f12ad8)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:24.168448 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d8df89-fa1d-43f3-a297-902c484facf2 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:24.170728 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 130/257] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:34:23 2022] GET /identity/v3/users/c6e0b984433648909afcb604f5f12ad8 => generated 297 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:24.176651 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d8df89-fa1d-43f3-a297-902c484facf2 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:24.274586 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cceba51-069a-4b2b-942a-872422065db9 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:24.290096 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:24.314705 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d8df89-fa1d-43f3-a297-902c484facf2 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:24.322253 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:24.346594 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd87e2f-d0ad-4d06-905c-c225ff2e776e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:24.347239 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd87e2f-d0ad-4d06-905c-c225ff2e776e None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:24.347511 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd87e2f-d0ad-4d06-905c-c225ff2e776e None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:24.347777 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd87e2f-d0ad-4d06-905c-c225ff2e776e None admin] PATH_INFO: `/v3/users/fa2b48749a564275b6ac6bed107a81a5` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:24.349154 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd87e2f-d0ad-4d06-905c-c225ff2e776e None admin] RBAC: Authorizing `identity:get_user(user_id=fa2b48749a564275b6ac6bed107a81a5)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:24.350796 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd87e2f-d0ad-4d06-905c-c225ff2e776e None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:24.352517 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 128/258] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:34:24 2022] GET /identity/v3/users/fa2b48749a564275b6ac6bed107a81a5 => generated 283 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:24.353306 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd87e2f-d0ad-4d06-905c-c225ff2e776e None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:24.353796 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765fbedb-6573-4af2-b3e8-094011b6b9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:24.354435 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765fbedb-6573-4af2-b3e8-094011b6b9cd None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:24.354712 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765fbedb-6573-4af2-b3e8-094011b6b9cd None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:24.355010 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765fbedb-6573-4af2-b3e8-094011b6b9cd None admin] PATH_INFO: `/v3/projects/d6cc740b2ba549809a3b7f863d43517c` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:24.356189 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765fbedb-6573-4af2-b3e8-094011b6b9cd None admin] RBAC: Authorizing `identity:get_project(project_id=d6cc740b2ba549809a3b7f863d43517c)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:24.358722 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765fbedb-6573-4af2-b3e8-094011b6b9cd None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:24.360759 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 131/259] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:24 2022] GET /identity/v3/projects/d6cc740b2ba549809a3b7f863d43517c => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:34:24.361596 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765fbedb-6573-4af2-b3e8-094011b6b9cd None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:24.486462 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765fbedb-6573-4af2-b3e8-094011b6b9cd None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:24.522579 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaca4f9-e21f-4a5f-a76e-a82259d0aed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:24.523285 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaca4f9-e21f-4a5f-a76e-a82259d0aed1 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:24.523563 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaca4f9-e21f-4a5f-a76e-a82259d0aed1 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:24.523838 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaca4f9-e21f-4a5f-a76e-a82259d0aed1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:24.585696 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:24.599130 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaca4f9-e21f-4a5f-a76e-a82259d0aed1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:24.600746 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaca4f9-e21f-4a5f-a76e-a82259d0aed1 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:24.716011 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 132/260] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:34:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:34:24.717015 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaca4f9-e21f-4a5f-a76e-a82259d0aed1 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:24.739202 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd87e2f-d0ad-4d06-905c-c225ff2e776e None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:24.809685 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:24.828775 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaca4f9-e21f-4a5f-a76e-a82259d0aed1 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:24.846518 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98f9f7-48a2-4d90-8022-6484def751c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:24.847799 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98f9f7-48a2-4d90-8022-6484def751c9 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:24.848072 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98f9f7-48a2-4d90-8022-6484def751c9 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:24.848334 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98f9f7-48a2-4d90-8022-6484def751c9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:24.868432 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed218716-85ac-4612-bfb7-2a19d4348694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:24.869249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed218716-85ac-4612-bfb7-2a19d4348694 None admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:24.869687 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed218716-85ac-4612-bfb7-2a19d4348694 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:24.870061 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed218716-85ac-4612-bfb7-2a19d4348694 None admin] PATH_INFO: `/v3/projects/d6cc740b2ba549809a3b7f863d43517c/users/c6e0b984433648909afcb604f5f12ad8/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:24.870358 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba98f9f7-48a2-4d90-8022-6484def751c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:24.872239 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed218716-85ac-4612-bfb7-2a19d4348694 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6cc740b2ba549809a3b7f863d43517c, user_id=c6e0b984433648909afcb604f5f12ad8, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:24.873819 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed218716-85ac-4612-bfb7-2a19d4348694 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:24.875422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-ba98f9f7-48a2-4d90-8022-6484def751c9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:34:24.875422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:24.875422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:24.875422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:24.875422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:24.875422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:24.875422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:24.875422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:24.875422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:24.875422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:24.875422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:24.875422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:24.875422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:24.875422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:24.875422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:24.875422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:24.876276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:24.876276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:24.876276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:24.876276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:24.876276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:24.876276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:24.876276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:24.876276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:24.876276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:24.876276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:24.876276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:24.876276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:24.876276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:24.876276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:24.876276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:24.876276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:24.877059 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:24.877059 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:24.877059 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:24.877059 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:24.877059 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:24.877059 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:24.877059 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:24.877059 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:24.877059 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:24.877059 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:24.877059 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:34:24.877059 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:24.877059 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 129/261] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 10 22:34:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 523 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:24.882541 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98f9f7-48a2-4d90-8022-6484def751c9 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:24.904772 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 133/262] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 10 22:34:24 2022] PUT /identity/v3/projects/d6cc740b2ba549809a3b7f863d43517c/users/c6e0b984433648909afcb604f5f12ad8/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:34:24.905815 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed218716-85ac-4612-bfb7-2a19d4348694 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:24.932933 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:25.054887 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98f9f7-48a2-4d90-8022-6484def751c9 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:25.138033 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8230d9-21a5-44e8-8a79-e2cb0ba5adea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:25.139288 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8230d9-21a5-44e8-8a79-e2cb0ba5adea None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:25.139511 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8230d9-21a5-44e8-8a79-e2cb0ba5adea None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:25.139766 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8230d9-21a5-44e8-8a79-e2cb0ba5adea None admin] PATH_INFO: `/v3/projects/d6cc740b2ba549809a3b7f863d43517c` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:25.143671 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8230d9-21a5-44e8-8a79-e2cb0ba5adea None admin] RBAC: Authorizing `identity:get_project(project_id=d6cc740b2ba549809a3b7f863d43517c)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:25.145120 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8230d9-21a5-44e8-8a79-e2cb0ba5adea None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:25.146932 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 130/263] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:24 2022] GET /identity/v3/projects/d6cc740b2ba549809a3b7f863d43517c => generated 302 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:25.151110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615ce7a-7e14-49ac-b3e4-f85d2653095b None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:25.151392 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615ce7a-7e14-49ac-b3e4-f85d2653095b None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:25.151667 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615ce7a-7e14-49ac-b3e4-f85d2653095b None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:25.153086 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 131/264] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:25.156424 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9615ce7a-7e14-49ac-b3e4-f85d2653095b None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:25.212822 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed218716-85ac-4612-bfb7-2a19d4348694 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:25.269467 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe6a97-a2b6-46af-8395-7b5a3ec37104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:25.270079 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe6a97-a2b6-46af-8395-7b5a3ec37104 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:25.270349 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe6a97-a2b6-46af-8395-7b5a3ec37104 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:25.270613 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe6a97-a2b6-46af-8395-7b5a3ec37104 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:25.272057 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe6a97-a2b6-46af-8395-7b5a3ec37104 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:25.272356 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe6a97-a2b6-46af-8395-7b5a3ec37104 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:25.273746 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe6a97-a2b6-46af-8395-7b5a3ec37104 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:25.277402 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:25.286968 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 134/265] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:34:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:25.288518 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba35b9-aaf8-42e1-b81e-26f2163bb73a None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:25.288798 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba35b9-aaf8-42e1-b81e-26f2163bb73a None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:25.289162 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba35b9-aaf8-42e1-b81e-26f2163bb73a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:25.317438 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-31ba35b9-aaf8-42e1-b81e-26f2163bb73a None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:25.318350 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9615ce7a-7e14-49ac-b3e4-f85d2653095b None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:25.318678 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-31ba35b9-aaf8-42e1-b81e-26f2163bb73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:25.320800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:25.357916 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1964f2-a15a-430b-89d7-52eaa8c9d505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:25.358533 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1964f2-a15a-430b-89d7-52eaa8c9d505 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:25.358809 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1964f2-a15a-430b-89d7-52eaa8c9d505 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:25.359076 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1964f2-a15a-430b-89d7-52eaa8c9d505 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:25.360047 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1964f2-a15a-430b-89d7-52eaa8c9d505 None admin] RBAC: Adding query filter params (role.id=6641e95fdc1f4c098ebc372bb692d227, scope.project.id=d6cc740b2ba549809a3b7f863d43517c, user.id=fa2b48749a564275b6ac6bed107a81a5) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:25.360361 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1964f2-a15a-430b-89d7-52eaa8c9d505 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:25.365680 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1964f2-a15a-430b-89d7-52eaa8c9d505 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:25.377377 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 132/266] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:34:25 2022] GET /identity/v3/role_assignments?user.id=fa2b48749a564275b6ac6bed107a81a5&scope.project.id=d6cc740b2ba549809a3b7f863d43517c&role.id=6641e95fdc1f4c098ebc372bb692d227 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:34:25.378273 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1964f2-a15a-430b-89d7-52eaa8c9d505 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:25.424486 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 135/267] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:25.469656 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba35b9-aaf8-42e1-b81e-26f2163bb73a None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:25.511764 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1964f2-a15a-430b-89d7-52eaa8c9d505 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:25.548525 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058b34c-8975-48b4-b6c1-3d830121fe93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:25.549158 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058b34c-8975-48b4-b6c1-3d830121fe93 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:25.549466 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058b34c-8975-48b4-b6c1-3d830121fe93 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:25.549731 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058b34c-8975-48b4-b6c1-3d830121fe93 None admin] PATH_INFO: `/v3/users/fa2b48749a564275b6ac6bed107a81a5` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:25.550972 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4058b34c-8975-48b4-b6c1-3d830121fe93 None admin] RBAC: Authorizing `identity:get_user(user_id=fa2b48749a564275b6ac6bed107a81a5)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:25.552475 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4058b34c-8975-48b4-b6c1-3d830121fe93 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:25.555331 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 133/268] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:34:25 2022] GET /identity/v3/users/fa2b48749a564275b6ac6bed107a81a5 => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:25.561243 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058b34c-8975-48b4-b6c1-3d830121fe93 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:25.608160 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:25.618841 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba35b9-aaf8-42e1-b81e-26f2163bb73a None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:25.654122 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:25.662064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b2204c-926e-40cf-8b0c-b8bfa64c2a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:25.662982 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2204c-926e-40cf-8b0c-b8bfa64c2a47 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:25.663516 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2204c-926e-40cf-8b0c-b8bfa64c2a47 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:25.663791 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2204c-926e-40cf-8b0c-b8bfa64c2a47 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:25.678107 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b2204c-926e-40cf-8b0c-b8bfa64c2a47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:25.679011 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058b34c-8975-48b4-b6c1-3d830121fe93 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:25.684154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-98b2204c-926e-40cf-8b0c-b8bfa64c2a47 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:25.684154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:25.684154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:25.684154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:25.684154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:25.684154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:25.684154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:25.684154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:25.684154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:25.684154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:25.684154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:25.684154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:25.684154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:25.684154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:25.684154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:25.684154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:25.685002 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:25.685002 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:25.685002 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:25.685002 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:25.685002 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:25.685002 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:25.685002 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:25.685002 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:25.685002 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:25.685002 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:25.685002 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:25.685002 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:25.685002 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:25.685002 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:25.685002 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:25.685002 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:25.685828 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:25.685828 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:25.685828 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:25.685828 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:25.685828 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:25.685828 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:25.685828 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:25.685828 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:25.685828 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:25.685828 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:25.685828 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:25.685828 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:25.685828 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 136/269] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:34:25.694084 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b2204c-926e-40cf-8b0c-b8bfa64c2a47 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:25.724066 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cbe13f-589d-4b76-a375-5de4ef6f2871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:25.724672 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cbe13f-589d-4b76-a375-5de4ef6f2871 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:25.724949 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cbe13f-589d-4b76-a375-5de4ef6f2871 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:25.725291 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cbe13f-589d-4b76-a375-5de4ef6f2871 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:25.808055 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cbe13f-589d-4b76-a375-5de4ef6f2871 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:25.809602 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cbe13f-589d-4b76-a375-5de4ef6f2871 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:25.831674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b2204c-926e-40cf-8b0c-b8bfa64c2a47 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:25.854922 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:25.889729 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2322464-8f8f-4643-8b6f-9cb905b699d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:25.890522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2322464-8f8f-4643-8b6f-9cb905b699d8 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:25.890798 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2322464-8f8f-4643-8b6f-9cb905b699d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:25.891636 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2322464-8f8f-4643-8b6f-9cb905b699d8 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:25.892643 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2322464-8f8f-4643-8b6f-9cb905b699d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:25.893408 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2322464-8f8f-4643-8b6f-9cb905b699d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:25.895067 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2322464-8f8f-4643-8b6f-9cb905b699d8 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:25.906639 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 137/270] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 10 22:34:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:34:25.908823 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0d427-68d5-4505-93c0-46af37f6031a None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:25.909101 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0d427-68d5-4505-93c0-46af37f6031a None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:25.910025 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0d427-68d5-4505-93c0-46af37f6031a None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:25.912640 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 138/271] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:25.913531 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0d427-68d5-4505-93c0-46af37f6031a None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:25.943691 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 134/272] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:34:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 22:34:25.949610 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae958382-1936-4391-8364-dcf536d88145 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:25.949890 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae958382-1936-4391-8364-dcf536d88145 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:25.950212 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae958382-1936-4391-8364-dcf536d88145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:25.968661 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-ae958382-1936-4391-8364-dcf536d88145 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:25.970099 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-ae958382-1936-4391-8364-dcf536d88145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:25.975923 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:26.032803 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0d427-68d5-4505-93c0-46af37f6031a None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:26.044454 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 135/273] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:26.045416 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae958382-1936-4391-8364-dcf536d88145 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:26.069347 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb7cc27-2e25-4554-bdac-8301fe8f2baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:26.070122 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7cc27-2e25-4554-bdac-8301fe8f2baa None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:26.070403 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7cc27-2e25-4554-bdac-8301fe8f2baa None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:26.070673 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7cc27-2e25-4554-bdac-8301fe8f2baa None admin] PATH_INFO: `/v3/users/df02ce4ed8cd42dbb7c019ffc03ac23e` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:26.089130 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb7cc27-2e25-4554-bdac-8301fe8f2baa None admin] RBAC: Authorizing `identity:get_user(user_id=df02ce4ed8cd42dbb7c019ffc03ac23e)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:26.091039 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb7cc27-2e25-4554-bdac-8301fe8f2baa None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:26.094595 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 139/274] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:34:25 2022] GET /identity/v3/users/df02ce4ed8cd42dbb7c019ffc03ac23e => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:26.095633 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb7cc27-2e25-4554-bdac-8301fe8f2baa None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:26.116139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:26.155804 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae958382-1936-4391-8364-dcf536d88145 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:26.194642 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a3f0e-1915-432a-97ef-5cce66f6690b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:26.195299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a3f0e-1915-432a-97ef-5cce66f6690b None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:26.195572 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a3f0e-1915-432a-97ef-5cce66f6690b None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:26.195833 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a3f0e-1915-432a-97ef-5cce66f6690b None admin] PATH_INFO: `/v3/projects/d6cc740b2ba549809a3b7f863d43517c` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:26.196997 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268a3f0e-1915-432a-97ef-5cce66f6690b None admin] RBAC: Authorizing `identity:get_project(project_id=d6cc740b2ba549809a3b7f863d43517c)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:26.198461 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268a3f0e-1915-432a-97ef-5cce66f6690b None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:26.205439 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 136/275] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:26 2022] GET /identity/v3/projects/d6cc740b2ba549809a3b7f863d43517c => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:26.206337 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a3f0e-1915-432a-97ef-5cce66f6690b None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:26.236486 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb7cc27-2e25-4554-bdac-8301fe8f2baa None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:26.246750 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:26.283519 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d0ddd-5d94-45be-b418-d996d4ac31ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:26.284153 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d0ddd-5d94-45be-b418-d996d4ac31ea None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:26.284432 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d0ddd-5d94-45be-b418-d996d4ac31ea None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:26.284693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d0ddd-5d94-45be-b418-d996d4ac31ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:26.285845 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d0ddd-5d94-45be-b418-d996d4ac31ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:26.287264 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d0ddd-5d94-45be-b418-d996d4ac31ea None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:26.289176 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 140/276] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:34:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:26.290195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d0ddd-5d94-45be-b418-d996d4ac31ea None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:26.320694 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a3f0e-1915-432a-97ef-5cce66f6690b None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:26.358881 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e18f2e6-79dc-4334-9516-ce2866aa88ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:26.359601 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18f2e6-79dc-4334-9516-ce2866aa88ef None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:26.359876 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18f2e6-79dc-4334-9516-ce2866aa88ef None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:26.360138 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18f2e6-79dc-4334-9516-ce2866aa88ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:26.360950 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e18f2e6-79dc-4334-9516-ce2866aa88ef None admin] RBAC: Adding query filter params (role.id=1085473c6e534f87812b05c7014f77c2, user.id=df02ce4ed8cd42dbb7c019ffc03ac23e, scope.system=all) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:26.361253 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e18f2e6-79dc-4334-9516-ce2866aa88ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:26.362771 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e18f2e6-79dc-4334-9516-ce2866aa88ef None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:26.364507 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:26.370730 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 137/277] 10.222.0.76 () {56 vars in 1308 bytes} [Wed Aug 10 22:34:26 2022] GET /identity/v3/role_assignments?user.id=df02ce4ed8cd42dbb7c019ffc03ac23e&scope.system=all&role.id=1085473c6e534f87812b05c7014f77c2 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:26.371539 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e18f2e6-79dc-4334-9516-ce2866aa88ef None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:26.396869 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d0ddd-5d94-45be-b418-d996d4ac31ea None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:26.447304 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7263e1-95b1-4d7b-8a84-79e0c9eb229d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:26.448054 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7263e1-95b1-4d7b-8a84-79e0c9eb229d None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:26.448332 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7263e1-95b1-4d7b-8a84-79e0c9eb229d None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:26.448597 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7263e1-95b1-4d7b-8a84-79e0c9eb229d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:26.449358 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7263e1-95b1-4d7b-8a84-79e0c9eb229d None admin] RBAC: Adding query filter params (role.id=e22e4d2b3d06480bbcfa4d2ee3387c90, scope.project.id=d6cc740b2ba549809a3b7f863d43517c, user.id=fa2b48749a564275b6ac6bed107a81a5) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:26.449663 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7263e1-95b1-4d7b-8a84-79e0c9eb229d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:26.451101 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7263e1-95b1-4d7b-8a84-79e0c9eb229d None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:26.466690 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 141/278] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:34:26 2022] GET /identity/v3/role_assignments?user.id=fa2b48749a564275b6ac6bed107a81a5&scope.project.id=d6cc740b2ba549809a3b7f863d43517c&role.id=e22e4d2b3d06480bbcfa4d2ee3387c90 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:26.495938 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b06b71-17e9-4867-9f94-ee423e0edc06 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:26.496246 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b06b71-17e9-4867-9f94-ee423e0edc06 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:26.496473 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b06b71-17e9-4867-9f94-ee423e0edc06 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:26.499600 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 142/279] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:34:26.509170 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e18f2e6-79dc-4334-9516-ce2866aa88ef None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:26.510762 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85803de6-e447-4ab4-82d6-2449530499aa None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:26.511043 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85803de6-e447-4ab4-82d6-2449530499aa None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:26.511318 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85803de6-e447-4ab4-82d6-2449530499aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:26.548803 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-85803de6-e447-4ab4-82d6-2449530499aa None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:26.550353 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-85803de6-e447-4ab4-82d6-2449530499aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:26.550733 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b716f50-f942-4c1f-b9d0-b1117cdf29da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:26.551355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b716f50-f942-4c1f-b9d0-b1117cdf29da None admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:26.551628 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b716f50-f942-4c1f-b9d0-b1117cdf29da None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:26.551894 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b716f50-f942-4c1f-b9d0-b1117cdf29da None admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:26.552660 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b716f50-f942-4c1f-b9d0-b1117cdf29da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:26.555069 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b716f50-f942-4c1f-b9d0-b1117cdf29da None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:26.585934 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 138/280] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 10 22:34:26 2022] POST /identity/v3/users => generated 293 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:34:26.621074 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:26.687799 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 143/281] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:26.736601 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b716f50-f942-4c1f-b9d0-b1117cdf29da None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:26.825711 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:26.874332 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b716f50-f942-4c1f-b9d0-b1117cdf29da None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:26.910114 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e0b3d-be59-4fb3-8508-1e6cb5c7ca70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:26.910741 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e0b3d-be59-4fb3-8508-1e6cb5c7ca70 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:26.911016 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e0b3d-be59-4fb3-8508-1e6cb5c7ca70 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:26.911281 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e0b3d-be59-4fb3-8508-1e6cb5c7ca70 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:26.923658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e0b3d-be59-4fb3-8508-1e6cb5c7ca70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:26.928740 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-473e0b3d-be59-4fb3-8508-1e6cb5c7ca70 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:34:26.928740 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:26.928740 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:26.928740 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:26.928740 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:26.928740 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:26.928740 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:26.928740 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:26.928740 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:26.928740 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:26.928740 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:26.928740 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:26.928740 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:26.928740 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:26.928740 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:26.928740 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:26.929656 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:26.929656 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:26.929656 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:26.929656 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:26.929656 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:26.929656 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:26.929656 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:26.929656 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:26.929656 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:26.929656 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:26.929656 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:26.929656 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:26.929656 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:26.929656 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:26.929656 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:26.929656 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:26.930699 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:26.930699 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:26.930699 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:26.930699 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:26.930699 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:26.930699 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:26.930699 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:26.930699 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:26.930699 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:26.930699 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:26.930699 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:34:26.930699 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:26.930699 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 139/282] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:26.937180 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85803de6-e447-4ab4-82d6-2449530499aa None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:27.064767 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85803de6-e447-4ab4-82d6-2449530499aa None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:27.108769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a431c-160e-4cd8-826d-c2ad2652fa43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:27.109422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a431c-160e-4cd8-826d-c2ad2652fa43 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:27.109700 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a431c-160e-4cd8-826d-c2ad2652fa43 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:27.109972 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a431c-160e-4cd8-826d-c2ad2652fa43 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:27.110666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a431c-160e-4cd8-826d-c2ad2652fa43 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:27.110965 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a431c-160e-4cd8-826d-c2ad2652fa43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:27.112402 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a431c-160e-4cd8-826d-c2ad2652fa43 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:27.121824 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 144/283] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 10 22:34:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:27.127894 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e0b3d-be59-4fb3-8508-1e6cb5c7ca70 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:27.239025 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e0b3d-be59-4fb3-8508-1e6cb5c7ca70 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:27.254053 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:27.280409 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44afde64-8de1-4c82-be55-6ca7c5dffac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:27.281054 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44afde64-8de1-4c82-be55-6ca7c5dffac2 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:27.281364 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44afde64-8de1-4c82-be55-6ca7c5dffac2 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:27.281635 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44afde64-8de1-4c82-be55-6ca7c5dffac2 None admin] PATH_INFO: `/v3/users/192102ee45ad40728357b262abd23a82` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:27.283452 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44afde64-8de1-4c82-be55-6ca7c5dffac2 None admin] RBAC: Authorizing `identity:get_user(user_id=192102ee45ad40728357b262abd23a82)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:27.284913 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44afde64-8de1-4c82-be55-6ca7c5dffac2 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:27.286898 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 140/284] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:34:27 2022] GET /identity/v3/users/192102ee45ad40728357b262abd23a82 => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:27.292852 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a431c-160e-4cd8-826d-c2ad2652fa43 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:27.337997 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:27.443622 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a431c-160e-4cd8-826d-c2ad2652fa43 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:27.499744 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117bfac-449c-46cd-ab92-1eaaae65bf46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:27.500371 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117bfac-449c-46cd-ab92-1eaaae65bf46 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:27.500641 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117bfac-449c-46cd-ab92-1eaaae65bf46 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:27.500904 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117bfac-449c-46cd-ab92-1eaaae65bf46 None admin] PATH_INFO: `/v3/projects/d6cc740b2ba549809a3b7f863d43517c` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:27.502053 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b117bfac-449c-46cd-ab92-1eaaae65bf46 None admin] RBAC: Authorizing `identity:get_project(project_id=d6cc740b2ba549809a3b7f863d43517c)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:27.503486 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b117bfac-449c-46cd-ab92-1eaaae65bf46 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:27.505402 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 145/285] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:27 2022] GET /identity/v3/projects/d6cc740b2ba549809a3b7f863d43517c => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:27.510962 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44afde64-8de1-4c82-be55-6ca7c5dffac2 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:27.632937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44afde64-8de1-4c82-be55-6ca7c5dffac2 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:27.673012 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d574f848-9db8-43c0-bc2b-0bec8eaab1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:27.673711 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574f848-9db8-43c0-bc2b-0bec8eaab1d0 None admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:27.673985 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574f848-9db8-43c0-bc2b-0bec8eaab1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:27.674252 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574f848-9db8-43c0-bc2b-0bec8eaab1d0 None admin] PATH_INFO: `/v3/projects/d6cc740b2ba549809a3b7f863d43517c/users/192102ee45ad40728357b262abd23a82/roles/05a9201bdeee4f2ab18c648d5e6cb250` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:27.676182 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d574f848-9db8-43c0-bc2b-0bec8eaab1d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6cc740b2ba549809a3b7f863d43517c, user_id=192102ee45ad40728357b262abd23a82, role_id=05a9201bdeee4f2ab18c648d5e6cb250)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:27.677654 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d574f848-9db8-43c0-bc2b-0bec8eaab1d0 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:27.705105 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 141/286] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 10 22:34:27 2022] PUT /identity/v3/projects/d6cc740b2ba549809a3b7f863d43517c/users/192102ee45ad40728357b262abd23a82/roles/05a9201bdeee4f2ab18c648d5e6cb250 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:34:27.771067 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5ca0e-ba79-46e6-8320-d4124f4a57c7 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:27.771390 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5ca0e-ba79-46e6-8320-d4124f4a57c7 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:27.771698 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5ca0e-ba79-46e6-8320-d4124f4a57c7 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:27.773355 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 146/287] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:27.778725 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741057b-84aa-4b6a-a8f0-a8e2b04c989d None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:27.779253 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741057b-84aa-4b6a-a8f0-a8e2b04c989d None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:27.779771 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741057b-84aa-4b6a-a8f0-a8e2b04c989d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:27.806893 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-4741057b-84aa-4b6a-a8f0-a8e2b04c989d None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:27.808472 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-4741057b-84aa-4b6a-a8f0-a8e2b04c989d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:27.862608 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:27.905327 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 142/288] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:27.938266 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba5ca0e-ba79-46e6-8320-d4124f4a57c7 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:28.068489 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:28.078629 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba5ca0e-ba79-46e6-8320-d4124f4a57c7 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:28.125704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b21ee9d-99ae-4bd6-b747-532197d6be70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:28.126342 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21ee9d-99ae-4bd6-b747-532197d6be70 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:28.126614 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21ee9d-99ae-4bd6-b747-532197d6be70 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:28.126943 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21ee9d-99ae-4bd6-b747-532197d6be70 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:28.142757 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b21ee9d-99ae-4bd6-b747-532197d6be70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:28.147504 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-8b21ee9d-99ae-4bd6-b747-532197d6be70 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:28.147504 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:28.147504 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:28.147504 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:28.147504 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:28.147504 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:28.147504 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:28.147504 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:28.147504 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:28.147504 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:28.147504 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:28.147504 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:28.147504 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:28.147504 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:28.147504 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:28.147504 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:28.148345 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:28.148345 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:28.148345 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:28.148345 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:28.148345 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:28.148345 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:28.148345 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:28.148345 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:28.148345 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:28.148345 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:28.148345 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:28.148345 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:28.148345 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:28.148345 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:28.148345 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:28.148345 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:28.149133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:28.149133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:28.149133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:28.149133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:28.149133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:28.149133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:28.149133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:28.149133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:28.149133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:28.149133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:28.149133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:28.149133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:28.149133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 147/289] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:28.155727 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4741057b-84aa-4b6a-a8f0-a8e2b04c989d None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:28.316551 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4741057b-84aa-4b6a-a8f0-a8e2b04c989d None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:28.367054 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9636da38-0ee9-42bc-a48f-2f7d3f82c22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:28.367668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636da38-0ee9-42bc-a48f-2f7d3f82c22f None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:28.367946 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636da38-0ee9-42bc-a48f-2f7d3f82c22f None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:28.368220 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636da38-0ee9-42bc-a48f-2f7d3f82c22f None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:28.368876 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9636da38-0ee9-42bc-a48f-2f7d3f82c22f None admin] RBAC: Adding query filter params (name=member) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:28.369174 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9636da38-0ee9-42bc-a48f-2f7d3f82c22f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:28.370579 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9636da38-0ee9-42bc-a48f-2f7d3f82c22f None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:28.385070 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 143/290] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 10 22:34:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:28.391188 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b21ee9d-99ae-4bd6-b747-532197d6be70 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:28.553998 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b21ee9d-99ae-4bd6-b747-532197d6be70 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:28.603632 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693838f-661e-44a2-8735-dd3d9949df2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:28.604303 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693838f-661e-44a2-8735-dd3d9949df2a None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:28.604581 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693838f-661e-44a2-8735-dd3d9949df2a None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:28.604893 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693838f-661e-44a2-8735-dd3d9949df2a None admin] PATH_INFO: `/v3/users/c6e0b984433648909afcb604f5f12ad8` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:28.606979 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2693838f-661e-44a2-8735-dd3d9949df2a None admin] RBAC: Authorizing `identity:get_user(user_id=c6e0b984433648909afcb604f5f12ad8)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:28.608478 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2693838f-661e-44a2-8735-dd3d9949df2a None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:28.610460 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 148/291] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:34:28 2022] GET /identity/v3/users/c6e0b984433648909afcb604f5f12ad8 => generated 297 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:28.615717 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9636da38-0ee9-42bc-a48f-2f7d3f82c22f None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:28.681399 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:28.782835 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9636da38-0ee9-42bc-a48f-2f7d3f82c22f None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:28.799406 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5b129-1dd0-4d4e-8361-5620afd4cd08 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:28.799733 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5b129-1dd0-4d4e-8361-5620afd4cd08 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:28.800012 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5b129-1dd0-4d4e-8361-5620afd4cd08 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:28.801496 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 149/292] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:34:28.807163 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65538057-7739-4ae9-a9a3-6145b8bb7c07 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:28.807450 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65538057-7739-4ae9-a9a3-6145b8bb7c07 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:28.807721 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65538057-7739-4ae9-a9a3-6145b8bb7c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:28.829346 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-65538057-7739-4ae9-a9a3-6145b8bb7c07 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:28.830553 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-65538057-7739-4ae9-a9a3-6145b8bb7c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:28.832180 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c1063-3750-4641-b364-6e7ecc5d4328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:28.833007 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c1063-3750-4641-b364-6e7ecc5d4328 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:28.833486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c1063-3750-4641-b364-6e7ecc5d4328 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:28.833920 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c1063-3750-4641-b364-6e7ecc5d4328 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:28.869684 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:28.958333 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 150/293] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:34:28.960382 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c1063-3750-4641-b364-6e7ecc5d4328 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:28.962461 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c1063-3750-4641-b364-6e7ecc5d4328 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:28.995929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:29.002369 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65538057-7739-4ae9-a9a3-6145b8bb7c07 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:29.134870 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65538057-7739-4ae9-a9a3-6145b8bb7c07 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:29.157266 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 144/294] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:34:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:34:29.163102 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c1063-3750-4641-b364-6e7ecc5d4328 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:29.184040 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a8b8b-98c1-4eeb-9358-880af0901da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:29.185033 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a8b8b-98c1-4eeb-9358-880af0901da6 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:29.185574 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a8b8b-98c1-4eeb-9358-880af0901da6 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:29.186021 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a8b8b-98c1-4eeb-9358-880af0901da6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:29.214296 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a8b8b-98c1-4eeb-9358-880af0901da6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:29.219045 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-b59a8b8b-98c1-4eeb-9358-880af0901da6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:29.219045 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:29.219045 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:29.219045 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:29.219045 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:29.219045 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:29.219045 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:29.219045 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:29.219045 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:29.219045 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:29.219045 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:29.219045 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:29.219045 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:29.219045 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:29.219045 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:29.219045 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:29.219916 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:29.219916 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:29.219916 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:29.219916 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:29.219916 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:29.219916 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:29.219916 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:29.219916 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:29.219916 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:29.219916 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:29.219916 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:29.219916 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:29.219916 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:29.219916 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:29.219916 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:29.219916 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:29.220709 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:29.220709 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:29.220709 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:29.220709 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:29.220709 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:29.220709 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:29.220709 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:29.220709 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:29.220709 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:29.220709 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:29.220709 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:29.220709 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:29.220709 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 151/295] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:34:29.230784 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a8b8b-98c1-4eeb-9358-880af0901da6 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:29.280276 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:29.307987 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c1063-3750-4641-b364-6e7ecc5d4328 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:29.393755 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a05e8a-eb67-4a4e-9516-e6c87e8b96ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:29.394403 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a05e8a-eb67-4a4e-9516-e6c87e8b96ff None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:29.394685 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a05e8a-eb67-4a4e-9516-e6c87e8b96ff None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:29.394957 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a05e8a-eb67-4a4e-9516-e6c87e8b96ff None admin] PATH_INFO: `/v3/projects/d6cc740b2ba549809a3b7f863d43517c` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:29.396124 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a05e8a-eb67-4a4e-9516-e6c87e8b96ff None admin] RBAC: Authorizing `identity:get_project(project_id=d6cc740b2ba549809a3b7f863d43517c)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:29.397605 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a05e8a-eb67-4a4e-9516-e6c87e8b96ff None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:29.399549 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 145/296] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:29 2022] GET /identity/v3/projects/d6cc740b2ba549809a3b7f863d43517c => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:34:29.405924 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a05e8a-eb67-4a4e-9516-e6c87e8b96ff None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:29.424231 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:29.454416 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a8b8b-98c1-4eeb-9358-880af0901da6 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:29.495520 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3be656-00f9-4c55-8f9d-0f11f0a5f8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:29.496054 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3be656-00f9-4c55-8f9d-0f11f0a5f8c5 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:29.496389 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3be656-00f9-4c55-8f9d-0f11f0a5f8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:29.496668 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3be656-00f9-4c55-8f9d-0f11f0a5f8c5 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:29.497367 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3be656-00f9-4c55-8f9d-0f11f0a5f8c5 None admin] RBAC: Adding query filter params (name=member) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:29.497671 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3be656-00f9-4c55-8f9d-0f11f0a5f8c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:29.499264 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3be656-00f9-4c55-8f9d-0f11f0a5f8c5 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:29.513336 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 152/297] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 10 22:34:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:29.514656 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3be656-00f9-4c55-8f9d-0f11f0a5f8c5 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:29.600274 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a05e8a-eb67-4a4e-9516-e6c87e8b96ff None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:29.619409 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:29.646032 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f57cce-3986-4d94-b53f-24db054ceb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:29.646954 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f57cce-3986-4d94-b53f-24db054ceb85 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:29.647241 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f57cce-3986-4d94-b53f-24db054ceb85 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:29.647509 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f57cce-3986-4d94-b53f-24db054ceb85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:29.648974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f57cce-3986-4d94-b53f-24db054ceb85 None admin] RBAC: Adding query filter params (role.id=1085473c6e534f87812b05c7014f77c2, scope.project.id=d6cc740b2ba549809a3b7f863d43517c, user.id=c6e0b984433648909afcb604f5f12ad8) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:29.649306 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f57cce-3986-4d94-b53f-24db054ceb85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:29.650736 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f57cce-3986-4d94-b53f-24db054ceb85 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:29.651198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3be656-00f9-4c55-8f9d-0f11f0a5f8c5 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:29.658861 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 146/298] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:34:29 2022] GET /identity/v3/role_assignments?user.id=c6e0b984433648909afcb604f5f12ad8&scope.project.id=d6cc740b2ba549809a3b7f863d43517c&role.id=1085473c6e534f87812b05c7014f77c2 => generated 608 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:29.660294 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64367b9a-29bf-4e2c-acf0-4817c9a41190 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:29.660572 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64367b9a-29bf-4e2c-acf0-4817c9a41190 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:29.660887 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64367b9a-29bf-4e2c-acf0-4817c9a41190 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:29.666193 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 147/299] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:29.669517 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded65576-f661-48a7-9022-09d60f2e5d33 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:29.669882 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded65576-f661-48a7-9022-09d60f2e5d33 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:29.670156 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded65576-f661-48a7-9022-09d60f2e5d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:29.694841 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-ded65576-f661-48a7-9022-09d60f2e5d33 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:29.696343 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-ded65576-f661-48a7-9022-09d60f2e5d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:29.704772 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7dc5f2-84b1-4559-91a5-894f017e3b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:29.705825 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7dc5f2-84b1-4559-91a5-894f017e3b23 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:29.707087 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7dc5f2-84b1-4559-91a5-894f017e3b23 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:29.707590 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7dc5f2-84b1-4559-91a5-894f017e3b23 None admin] PATH_INFO: `/v3/users/df02ce4ed8cd42dbb7c019ffc03ac23e` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:29.711186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7dc5f2-84b1-4559-91a5-894f017e3b23 None admin] RBAC: Authorizing `identity:get_user(user_id=df02ce4ed8cd42dbb7c019ffc03ac23e)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:29.712672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7dc5f2-84b1-4559-91a5-894f017e3b23 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:29.714743 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 153/300] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:34:29 2022] GET /identity/v3/users/df02ce4ed8cd42dbb7c019ffc03ac23e => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:34:29.716188 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedeb8cf-86e7-440d-8145-e94b70074290 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:29.716464 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedeb8cf-86e7-440d-8145-e94b70074290 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:29.716730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedeb8cf-86e7-440d-8145-e94b70074290 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:29.719168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 154/301] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:29.721311 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedeb8cf-86e7-440d-8145-e94b70074290 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:29.780456 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:29.797683 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 148/302] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:29.799668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0198d-8c76-4159-9154-63d2a7df07b0 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:29.800138 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0198d-8c76-4159-9154-63d2a7df07b0 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:29.800569 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0198d-8c76-4159-9154-63d2a7df07b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:29.828580 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-90b0198d-8c76-4159-9154-63d2a7df07b0 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:29.830143 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-90b0198d-8c76-4159-9154-63d2a7df07b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:29.845084 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedeb8cf-86e7-440d-8145-e94b70074290 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:29.891366 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9141c2ed-d0ab-411e-a1be-83716f7d29cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:29.891366 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141c2ed-d0ab-411e-a1be-83716f7d29cc None admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:29.891366 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141c2ed-d0ab-411e-a1be-83716f7d29cc None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:29.891712 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141c2ed-d0ab-411e-a1be-83716f7d29cc None admin] PATH_INFO: `/v3/system/users/df02ce4ed8cd42dbb7c019ffc03ac23e/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:29.896469 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9141c2ed-d0ab-411e-a1be-83716f7d29cc None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=df02ce4ed8cd42dbb7c019ffc03ac23e, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:29.896469 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9141c2ed-d0ab-411e-a1be-83716f7d29cc None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:29.908474 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 155/303] 10.222.0.76 () {58 vars in 1334 bytes} [Wed Aug 10 22:34:29 2022] PUT /identity/v3/system/users/df02ce4ed8cd42dbb7c019ffc03ac23e/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:34:29.910535 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9141c2ed-d0ab-411e-a1be-83716f7d29cc None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:29.916916 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 149/304] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:29.934969 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:29.954745 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b0198d-8c76-4159-9154-63d2a7df07b0 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:30.055078 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9141c2ed-d0ab-411e-a1be-83716f7d29cc None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:30.099202 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b0198d-8c76-4159-9154-63d2a7df07b0 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:30.105587 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd0aa7-e34f-4918-9ad3-f309deacf77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:30.106479 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd0aa7-e34f-4918-9ad3-f309deacf77b None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:30.106761 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd0aa7-e34f-4918-9ad3-f309deacf77b None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:30.107033 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd0aa7-e34f-4918-9ad3-f309deacf77b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:30.126221 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd0aa7-e34f-4918-9ad3-f309deacf77b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:30.133007 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-62fd0aa7-e34f-4918-9ad3-f309deacf77b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:34:30.133007 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:30.133007 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:30.133007 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:30.133007 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:30.133007 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:30.133007 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:30.133007 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:30.133007 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:30.133007 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:30.133007 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:30.133007 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:30.133007 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:30.133007 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:30.133007 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:30.133007 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:30.134028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:30.134028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:30.134028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:30.134028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:30.134028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:30.134028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:30.134028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:30.134028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:30.134028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:30.134028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:30.134028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:30.134028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:30.134028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:30.134028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:30.134028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:30.134028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:30.134897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:30.134897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:30.134897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:30.134897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:30.134897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:30.134897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:30.134897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:30.134897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:30.134897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:30.134897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:30.134897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:34:30.134897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:30.135529 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:30.137913 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 156/305] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:30.146309 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd0aa7-e34f-4918-9ad3-f309deacf77b None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:30.150003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51f1a4-e9a8-45ff-89d6-ea4a9ac93efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:30.150546 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51f1a4-e9a8-45ff-89d6-ea4a9ac93efe None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:30.150857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51f1a4-e9a8-45ff-89d6-ea4a9ac93efe None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:30.151134 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51f1a4-e9a8-45ff-89d6-ea4a9ac93efe None admin] PATH_INFO: `/v3/domains/default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:30.152630 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b51f1a4-e9a8-45ff-89d6-ea4a9ac93efe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:30.153773 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b51f1a4-e9a8-45ff-89d6-ea4a9ac93efe None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:30.155660 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 150/306] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:34:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:30.161870 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51f1a4-e9a8-45ff-89d6-ea4a9ac93efe None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:30.244517 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:30.276312 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51f1a4-e9a8-45ff-89d6-ea4a9ac93efe None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:30.287923 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd0aa7-e34f-4918-9ad3-f309deacf77b None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:30.315890 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf93f85-0968-4a7d-b2a0-b13e77d55158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:30.316420 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf93f85-0968-4a7d-b2a0-b13e77d55158 None admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:30.316701 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf93f85-0968-4a7d-b2a0-b13e77d55158 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:30.316977 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf93f85-0968-4a7d-b2a0-b13e77d55158 None admin] PATH_INFO: `/v3/groups` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:30.317634 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf93f85-0968-4a7d-b2a0-b13e77d55158 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:30.319413 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf93f85-0968-4a7d-b2a0-b13e77d55158 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:30.323774 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928786a-b1eb-4b31-8376-6c2b9e594587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:30.324381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928786a-b1eb-4b31-8376-6c2b9e594587 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:30.324659 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928786a-b1eb-4b31-8376-6c2b9e594587 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:30.324930 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928786a-b1eb-4b31-8376-6c2b9e594587 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:30.325624 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928786a-b1eb-4b31-8376-6c2b9e594587 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:30.325925 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928786a-b1eb-4b31-8376-6c2b9e594587 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:30.327402 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928786a-b1eb-4b31-8376-6c2b9e594587 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:30.341533 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 157/307] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 10 22:34:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:30.341533 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42251d72-7ef3-4269-a0dd-20879de4d4c6 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:30.341533 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42251d72-7ef3-4269-a0dd-20879de4d4c6 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:30.341533 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42251d72-7ef3-4269-a0dd-20879de4d4c6 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:30.341533 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 158/308] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:30.345423 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42251d72-7ef3-4269-a0dd-20879de4d4c6 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:30.345594 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 151/309] 10.222.0.76 () {60 vars in 1134 bytes} [Wed Aug 10 22:34:30 2022] POST /identity/v3/groups => generated 229 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:34:30.353556 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043936c5-34da-49c4-836d-cfd54ec99111 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:30.354052 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043936c5-34da-49c4-836d-cfd54ec99111 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:30.354472 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043936c5-34da-49c4-836d-cfd54ec99111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:30.376546 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-043936c5-34da-49c4-836d-cfd54ec99111 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:30.378201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-043936c5-34da-49c4-836d-cfd54ec99111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:30.444529 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:30.473387 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 152/310] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:34:30.492888 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043936c5-34da-49c4-836d-cfd54ec99111 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:30.502640 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42251d72-7ef3-4269-a0dd-20879de4d4c6 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:30.536600 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:30.543898 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edfd6e5-aae7-4326-b5f8-7390fa2d6c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:30.544636 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edfd6e5-aae7-4326-b5f8-7390fa2d6c2b None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:30.544995 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edfd6e5-aae7-4326-b5f8-7390fa2d6c2b None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:30.545364 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edfd6e5-aae7-4326-b5f8-7390fa2d6c2b None admin] PATH_INFO: `/v3/users/62cfebaddb434256800d5f1f67673721` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:30.561345 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edfd6e5-aae7-4326-b5f8-7390fa2d6c2b None admin] RBAC: Authorizing `identity:get_user(user_id=62cfebaddb434256800d5f1f67673721)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:30.562875 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edfd6e5-aae7-4326-b5f8-7390fa2d6c2b None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:30.564694 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 159/311] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:34:30 2022] GET /identity/v3/users/62cfebaddb434256800d5f1f67673721 => generated 293 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:30.572726 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edfd6e5-aae7-4326-b5f8-7390fa2d6c2b None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:30.624564 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043936c5-34da-49c4-836d-cfd54ec99111 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:30.669984 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d5fdb2-6e41-4a45-94ba-e41638b21c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:30.671034 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d5fdb2-6e41-4a45-94ba-e41638b21c7c None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:30.671553 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d5fdb2-6e41-4a45-94ba-e41638b21c7c None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:30.672025 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d5fdb2-6e41-4a45-94ba-e41638b21c7c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:30.685553 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edfd6e5-aae7-4326-b5f8-7390fa2d6c2b None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:30.688726 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d5fdb2-6e41-4a45-94ba-e41638b21c7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:30.695132 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-27d5fdb2-6e41-4a45-94ba-e41638b21c7c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:34:30.695132 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:30.695132 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:30.695132 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:30.695132 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:30.695132 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:30.695132 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:30.695132 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:30.695132 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:30.695132 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:30.695132 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:30.695132 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:30.695132 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:30.695132 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:30.695132 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:30.695132 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:30.696064 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:30.696064 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:30.696064 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:30.696064 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:30.696064 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:30.696064 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:30.696064 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:30.696064 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:30.696064 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:30.696064 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:30.696064 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:30.696064 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:30.696064 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:30.696064 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:30.696064 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:30.696064 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:30.696838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:30.696838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:30.696838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:30.696838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:30.696838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:30.696838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:30.696838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:30.696838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:30.696838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:30.696838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:30.696838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:34:30.696838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:30.698623 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 153/312] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:34:30.705586 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d5fdb2-6e41-4a45-94ba-e41638b21c7c None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:30.729720 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b29a7-b56c-4f35-a447-efb3ac24be16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:30.730689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b29a7-b56c-4f35-a447-efb3ac24be16 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:30.731238 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b29a7-b56c-4f35-a447-efb3ac24be16 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:30.731860 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b29a7-b56c-4f35-a447-efb3ac24be16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:30.732939 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17b29a7-b56c-4f35-a447-efb3ac24be16 None admin] RBAC: Adding query filter params (role.id=05a9201bdeee4f2ab18c648d5e6cb250, user.id=62cfebaddb434256800d5f1f67673721, scope.system=all) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:30.733485 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17b29a7-b56c-4f35-a447-efb3ac24be16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:30.735419 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17b29a7-b56c-4f35-a447-efb3ac24be16 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:30.742366 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 160/313] 10.222.0.76 () {56 vars in 1308 bytes} [Wed Aug 10 22:34:30 2022] GET /identity/v3/role_assignments?user.id=62cfebaddb434256800d5f1f67673721&scope.system=all&role.id=05a9201bdeee4f2ab18c648d5e6cb250 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:34:30.760427 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:30.826858 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d5fdb2-6e41-4a45-94ba-e41638b21c7c None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:30.868252 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e885b-b248-45de-a021-e1b683c8741d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:30.869309 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e885b-b248-45de-a021-e1b683c8741d None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:30.869796 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e885b-b248-45de-a021-e1b683c8741d None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:30.870267 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e885b-b248-45de-a021-e1b683c8741d None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:30.871175 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e885b-b248-45de-a021-e1b683c8741d None admin] RBAC: Adding query filter params (name=reader) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:30.871680 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e885b-b248-45de-a021-e1b683c8741d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:30.873303 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e885b-b248-45de-a021-e1b683c8741d None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:30.884551 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 154/314] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 10 22:34:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:34:30.893333 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b29a7-b56c-4f35-a447-efb3ac24be16 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:30.955302 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:31.005767 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b29a7-b56c-4f35-a447-efb3ac24be16 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:31.044827 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4394e34-c764-49f7-8f1d-e842b51037b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:31.045496 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4394e34-c764-49f7-8f1d-e842b51037b5 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:31.045773 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4394e34-c764-49f7-8f1d-e842b51037b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:31.046040 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4394e34-c764-49f7-8f1d-e842b51037b5 None admin] PATH_INFO: `/v3/users/192102ee45ad40728357b262abd23a82` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:31.047180 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4394e34-c764-49f7-8f1d-e842b51037b5 None admin] RBAC: Authorizing `identity:get_user(user_id=192102ee45ad40728357b262abd23a82)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:31.048605 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4394e34-c764-49f7-8f1d-e842b51037b5 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:31.051341 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 161/315] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:34:30 2022] GET /identity/v3/users/192102ee45ad40728357b262abd23a82 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:34:31.056891 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e885b-b248-45de-a021-e1b683c8741d None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:31.167673 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e885b-b248-45de-a021-e1b683c8741d None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:31.176119 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:31.207048 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aef717-b8ab-4be1-afe5-f93918b074c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:31.208019 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aef717-b8ab-4be1-afe5-f93918b074c5 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:31.208567 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aef717-b8ab-4be1-afe5-f93918b074c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:31.209041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aef717-b8ab-4be1-afe5-f93918b074c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:31.284281 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aef717-b8ab-4be1-afe5-f93918b074c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:31.286133 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aef717-b8ab-4be1-afe5-f93918b074c5 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:31.393729 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 155/316] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:34:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:34:31.405309 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4394e34-c764-49f7-8f1d-e842b51037b5 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:31.517010 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4394e34-c764-49f7-8f1d-e842b51037b5 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:31.540855 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:31.553573 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9824a-6ec5-48cb-b921-90c9a74163c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:31.554208 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9824a-6ec5-48cb-b921-90c9a74163c8 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:31.554482 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9824a-6ec5-48cb-b921-90c9a74163c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:31.554750 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9824a-6ec5-48cb-b921-90c9a74163c8 None admin] PATH_INFO: `/v3/projects/d6cc740b2ba549809a3b7f863d43517c` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:31.555872 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb9824a-6ec5-48cb-b921-90c9a74163c8 None admin] RBAC: Authorizing `identity:get_project(project_id=d6cc740b2ba549809a3b7f863d43517c)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:31.557336 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb9824a-6ec5-48cb-b921-90c9a74163c8 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:31.559256 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 162/317] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:31 2022] GET /identity/v3/projects/d6cc740b2ba549809a3b7f863d43517c => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:31.566372 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aef717-b8ab-4be1-afe5-f93918b074c5 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:31.585644 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:31.677471 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aef717-b8ab-4be1-afe5-f93918b074c5 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:31.715781 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73158c05-bc44-4379-9fa3-0bebee43f71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:31.717016 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73158c05-bc44-4379-9fa3-0bebee43f71e None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:31.717515 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73158c05-bc44-4379-9fa3-0bebee43f71e None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:31.717989 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73158c05-bc44-4379-9fa3-0bebee43f71e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:31.718888 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73158c05-bc44-4379-9fa3-0bebee43f71e None admin] RBAC: Adding query filter params (role.id=05a9201bdeee4f2ab18c648d5e6cb250, scope.project.id=d6cc740b2ba549809a3b7f863d43517c, user.id=192102ee45ad40728357b262abd23a82) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:31.719373 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73158c05-bc44-4379-9fa3-0bebee43f71e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:31.720997 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73158c05-bc44-4379-9fa3-0bebee43f71e None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:31.727605 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 156/318] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:34:31 2022] GET /identity/v3/role_assignments?user.id=192102ee45ad40728357b262abd23a82&scope.project.id=d6cc740b2ba549809a3b7f863d43517c&role.id=05a9201bdeee4f2ab18c648d5e6cb250 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:32.010381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53a069-6fb5-4352-9b95-f9c6b5d6b062 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:32.010600 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53a069-6fb5-4352-9b95-f9c6b5d6b062 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:32.011004 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53a069-6fb5-4352-9b95-f9c6b5d6b062 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:32.012517 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 163/319] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:32.017981 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296712e8-52e9-4371-a466-b0815feb6da7 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:32.018268 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296712e8-52e9-4371-a466-b0815feb6da7 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:32.018541 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296712e8-52e9-4371-a466-b0815feb6da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:32.036568 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-296712e8-52e9-4371-a466-b0815feb6da7 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:32.037775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-296712e8-52e9-4371-a466-b0815feb6da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:32.101386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:32.108200 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 157/320] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:32.131465 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe53a069-6fb5-4352-9b95-f9c6b5d6b062 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:32.236515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe53a069-6fb5-4352-9b95-f9c6b5d6b062 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:32.271901 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72747ff-5107-492a-9e8b-9176f88226dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:32.272448 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72747ff-5107-492a-9e8b-9176f88226dd None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:32.272722 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72747ff-5107-492a-9e8b-9176f88226dd None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:32.272989 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72747ff-5107-492a-9e8b-9176f88226dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:32.283729 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72747ff-5107-492a-9e8b-9176f88226dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:32.288395 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-c72747ff-5107-492a-9e8b-9176f88226dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:32.288395 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:32.288395 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:32.288395 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:32.288395 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:32.288395 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:32.288395 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:32.288395 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:32.288395 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:32.288395 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:32.288395 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:32.288395 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:32.288395 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:32.288395 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:32.288395 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:32.288395 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:32.289255 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:32.289255 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:32.289255 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:32.289255 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:32.289255 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:32.289255 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:32.289255 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:32.289255 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:32.289255 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:32.289255 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:32.289255 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:32.289255 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:32.289255 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:32.289255 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:32.289255 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:32.289255 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:32.290985 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:32.290985 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:32.290985 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:32.290985 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:32.290985 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:32.290985 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:32.290985 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:32.290985 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:32.290985 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:32.290985 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:32.290985 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:32.290985 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:32.292061 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 164/321] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:32.295917 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296712e8-52e9-4371-a466-b0815feb6da7 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:32.400185 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296712e8-52e9-4371-a466-b0815feb6da7 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:32.443117 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06b54d3-35b5-4ba8-bdf7-61b4c55b579d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:32.443737 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b54d3-35b5-4ba8-bdf7-61b4c55b579d None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:32.444011 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b54d3-35b5-4ba8-bdf7-61b4c55b579d None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:32.444282 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b54d3-35b5-4ba8-bdf7-61b4c55b579d None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:32.445517 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06b54d3-35b5-4ba8-bdf7-61b4c55b579d None admin] RBAC: Adding query filter params (name=member) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:32.445824 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06b54d3-35b5-4ba8-bdf7-61b4c55b579d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:32.447190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06b54d3-35b5-4ba8-bdf7-61b4c55b579d None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:32.455621 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 158/322] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 10 22:34:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:34:32.461103 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72747ff-5107-492a-9e8b-9176f88226dd None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:32.513613 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:32.563126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da6bfe-6494-4cbc-a1e7-bc33363c4778 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:32.563358 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da6bfe-6494-4cbc-a1e7-bc33363c4778 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:32.563603 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da6bfe-6494-4cbc-a1e7-bc33363c4778 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:32.565064 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 159/323] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:34:32.565687 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72747ff-5107-492a-9e8b-9176f88226dd None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:32.570350 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af5cc9-5e3b-45fc-946e-86a6d037f95b None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:32.570350 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af5cc9-5e3b-45fc-946e-86a6d037f95b None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:32.570538 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af5cc9-5e3b-45fc-946e-86a6d037f95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:32.587943 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-30af5cc9-5e3b-45fc-946e-86a6d037f95b None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:32.589139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-30af5cc9-5e3b-45fc-946e-86a6d037f95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:32.601880 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d6b7f-d08b-406c-a392-9fafe4079b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:32.602520 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d6b7f-d08b-406c-a392-9fafe4079b0d None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:32.602795 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d6b7f-d08b-406c-a392-9fafe4079b0d None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:32.603062 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d6b7f-d08b-406c-a392-9fafe4079b0d None admin] PATH_INFO: `/v3/users/df02ce4ed8cd42dbb7c019ffc03ac23e` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:32.604265 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d6b7f-d08b-406c-a392-9fafe4079b0d None admin] RBAC: Authorizing `identity:get_user(user_id=df02ce4ed8cd42dbb7c019ffc03ac23e)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:32.605792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d6b7f-d08b-406c-a392-9fafe4079b0d None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:32.607532 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 165/324] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:34:32 2022] GET /identity/v3/users/df02ce4ed8cd42dbb7c019ffc03ac23e => generated 293 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:32.612887 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d6b7f-d08b-406c-a392-9fafe4079b0d None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:32.663575 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:32.677322 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 160/325] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:34:32.697024 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af5cc9-5e3b-45fc-946e-86a6d037f95b None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:32.728602 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d6b7f-d08b-406c-a392-9fafe4079b0d None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:32.748528 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:32.767248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50578bd4-d0f5-4c52-ad04-fcf3f8b1c3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:32.767901 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50578bd4-d0f5-4c52-ad04-fcf3f8b1c3c7 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:32.768130 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50578bd4-d0f5-4c52-ad04-fcf3f8b1c3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:32.768524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50578bd4-d0f5-4c52-ad04-fcf3f8b1c3c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:32.769117 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50578bd4-d0f5-4c52-ad04-fcf3f8b1c3c7 None admin] RBAC: Adding query filter params (role.id=1085473c6e534f87812b05c7014f77c2, user.id=df02ce4ed8cd42dbb7c019ffc03ac23e, scope.system=all) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:32.769468 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50578bd4-d0f5-4c52-ad04-fcf3f8b1c3c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:32.771078 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50578bd4-d0f5-4c52-ad04-fcf3f8b1c3c7 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:32.782947 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 166/326] 10.222.0.76 () {56 vars in 1308 bytes} [Wed Aug 10 22:34:32 2022] GET /identity/v3/role_assignments?user.id=df02ce4ed8cd42dbb7c019ffc03ac23e&scope.system=all&role.id=1085473c6e534f87812b05c7014f77c2 => generated 510 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:34:32.803777 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af5cc9-5e3b-45fc-946e-86a6d037f95b None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:32.843484 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2148b8-568c-4ec4-baa4-2ef3f2f890bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:32.844204 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2148b8-568c-4ec4-baa4-2ef3f2f890bf None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:32.844480 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2148b8-568c-4ec4-baa4-2ef3f2f890bf None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:32.844753 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2148b8-568c-4ec4-baa4-2ef3f2f890bf None admin] PATH_INFO: `/v3/domains/default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:32.845971 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2148b8-568c-4ec4-baa4-2ef3f2f890bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:32.847495 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2148b8-568c-4ec4-baa4-2ef3f2f890bf None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:32.849491 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 161/327] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:34:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:34:32.854460 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f04717-3263-43e9-9706-a507cdbc3833 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:32.854754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f04717-3263-43e9-9706-a507cdbc3833 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:32.855016 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f04717-3263-43e9-9706-a507cdbc3833 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:32.855330 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2148b8-568c-4ec4-baa4-2ef3f2f890bf None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:32.856483 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 167/328] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:32.861967 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5af8fd-83ab-4fd0-9303-2bc420f02923 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:32.862237 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5af8fd-83ab-4fd0-9303-2bc420f02923 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:32.862510 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5af8fd-83ab-4fd0-9303-2bc420f02923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:32.881699 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-8f5af8fd-83ab-4fd0-9303-2bc420f02923 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:32.882944 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-8f5af8fd-83ab-4fd0-9303-2bc420f02923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:32.928953 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:32.960268 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 168/329] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:34:32.977797 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2148b8-568c-4ec4-baa4-2ef3f2f890bf None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:32.982937 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5af8fd-83ab-4fd0-9303-2bc420f02923 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:33.017747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7236762-e963-48f2-a92c-01ef71c19c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:33.018360 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7236762-e963-48f2-a92c-01ef71c19c6f None admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:33.018643 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7236762-e963-48f2-a92c-01ef71c19c6f None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:33.018918 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7236762-e963-48f2-a92c-01ef71c19c6f None admin] PATH_INFO: `/v3/groups` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:33.020957 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7236762-e963-48f2-a92c-01ef71c19c6f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:33.022034 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7236762-e963-48f2-a92c-01ef71c19c6f None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:33.038855 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 162/330] 10.222.0.76 () {60 vars in 1134 bytes} [Wed Aug 10 22:34:32 2022] POST /identity/v3/groups => generated 226 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:34:33.095044 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5af8fd-83ab-4fd0-9303-2bc420f02923 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:33.103076 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:33.132384 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867fb8e9-9156-42f0-a98b-b508b541657b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:33.133029 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867fb8e9-9156-42f0-a98b-b508b541657b None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:33.133343 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867fb8e9-9156-42f0-a98b-b508b541657b None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:33.133613 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867fb8e9-9156-42f0-a98b-b508b541657b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:33.144778 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867fb8e9-9156-42f0-a98b-b508b541657b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:33.149519 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-867fb8e9-9156-42f0-a98b-b508b541657b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:34:33.149519 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:33.149519 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:33.149519 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:33.149519 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:33.149519 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:33.149519 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:33.149519 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:33.149519 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:33.149519 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:33.149519 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:33.149519 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:33.149519 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:33.149519 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:33.149519 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:33.149519 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:33.150449 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:33.150449 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:33.150449 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:33.150449 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:33.150449 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:33.150449 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:33.150449 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:33.150449 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:33.150449 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:33.150449 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:33.150449 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:33.150449 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:33.150449 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:33.150449 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:33.150449 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:33.150449 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:33.151212 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:33.151212 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:33.151212 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:33.151212 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:33.151212 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:33.151212 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:33.151212 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:33.151212 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:33.151212 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:33.151212 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:33.151212 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:34:33.151212 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:33.151212 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 169/331] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:34:33.156800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7236762-e963-48f2-a92c-01ef71c19c6f None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:33.266177 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7236762-e963-48f2-a92c-01ef71c19c6f None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:33.288340 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:33.304686 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f780d-c1a6-425b-9896-d8e86ca88ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:33.305338 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f780d-c1a6-425b-9896-d8e86ca88ef6 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:33.305617 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f780d-c1a6-425b-9896-d8e86ca88ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:33.305893 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f780d-c1a6-425b-9896-d8e86ca88ef6 None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:33.306559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9f780d-c1a6-425b-9896-d8e86ca88ef6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:33.306863 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9f780d-c1a6-425b-9896-d8e86ca88ef6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:33.308242 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9f780d-c1a6-425b-9896-d8e86ca88ef6 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:33.317758 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 163/332] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 10 22:34:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:33.323360 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867fb8e9-9156-42f0-a98b-b508b541657b None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:33.434150 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867fb8e9-9156-42f0-a98b-b508b541657b None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:33.472606 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f00d6-8d37-46f2-8272-ffc625899a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:33.473240 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f00d6-8d37-46f2-8272-ffc625899a09 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:33.473749 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f00d6-8d37-46f2-8272-ffc625899a09 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:33.473865 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f00d6-8d37-46f2-8272-ffc625899a09 None admin] PATH_INFO: `/v3/users/62cfebaddb434256800d5f1f67673721` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:33.475043 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f00d6-8d37-46f2-8272-ffc625899a09 None admin] RBAC: Authorizing `identity:get_user(user_id=62cfebaddb434256800d5f1f67673721)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:33.476589 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f00d6-8d37-46f2-8272-ffc625899a09 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:33.478539 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 170/333] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:34:33 2022] GET /identity/v3/users/62cfebaddb434256800d5f1f67673721 => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:33.484422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f780d-c1a6-425b-9896-d8e86ca88ef6 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:33.599264 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f780d-c1a6-425b-9896-d8e86ca88ef6 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:33.635943 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29adbd8-a446-4cea-ab8e-072409420bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:33.636704 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29adbd8-a446-4cea-ab8e-072409420bf3 None admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:33.636989 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29adbd8-a446-4cea-ab8e-072409420bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:33.637294 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29adbd8-a446-4cea-ab8e-072409420bf3 None admin] PATH_INFO: `/v3/system/users/62cfebaddb434256800d5f1f67673721/roles/05a9201bdeee4f2ab18c648d5e6cb250` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:33.638756 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29adbd8-a446-4cea-ab8e-072409420bf3 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=62cfebaddb434256800d5f1f67673721, role_id=05a9201bdeee4f2ab18c648d5e6cb250)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:33.640153 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29adbd8-a446-4cea-ab8e-072409420bf3 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:33.652721 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 164/334] 10.222.0.76 () {58 vars in 1334 bytes} [Wed Aug 10 22:34:33 2022] PUT /identity/v3/system/users/62cfebaddb434256800d5f1f67673721/roles/05a9201bdeee4f2ab18c648d5e6cb250 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:34:35.429559 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15326e-6151-48ce-8002-68421dd992a7 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:35.430438 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15326e-6151-48ce-8002-68421dd992a7 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:35.430848 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15326e-6151-48ce-8002-68421dd992a7 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:35.432318 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 171/335] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:35.438014 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c94049-5468-4943-abbb-9f25fecbf322 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:35.438427 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c94049-5468-4943-abbb-9f25fecbf322 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:35.438628 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c94049-5468-4943-abbb-9f25fecbf322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:35.460828 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-83c94049-5468-4943-abbb-9f25fecbf322 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:35.462077 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-83c94049-5468-4943-abbb-9f25fecbf322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:35.527596 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:35.557846 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 165/336] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:35.596419 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f15326e-6151-48ce-8002-68421dd992a7 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:35.649345 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:35.744323 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f15326e-6151-48ce-8002-68421dd992a7 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:35.788193 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a5df6-b4d2-4920-b09f-54cc7299dbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:35.788835 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a5df6-b4d2-4920-b09f-54cc7299dbdc None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:35.789116 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a5df6-b4d2-4920-b09f-54cc7299dbdc None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:35.789429 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a5df6-b4d2-4920-b09f-54cc7299dbdc None admin] PATH_INFO: `/v3/roles/member` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:35.807378 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1a5df6-b4d2-4920-b09f-54cc7299dbdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:35.812106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-da1a5df6-b4d2-4920-b09f-54cc7299dbdc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:35.812106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:35.812106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:35.812106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:35.812106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:35.812106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:35.812106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:35.812106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:35.812106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:35.812106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:35.812106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:35.812106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:35.812106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:35.812106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:35.812106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:35.812106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:35.812951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:35.812951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:35.812951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:35.812951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:35.812951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:35.812951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:35.812951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:35.812951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:35.812951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:35.812951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:35.812951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:35.812951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:35.812951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:35.812951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:35.812951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:35.812951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:35.813857 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:35.813857 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:35.813857 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:35.813857 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:35.813857 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:35.813857 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:35.813857 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:35.813857 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:35.813857 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:35.813857 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:35.813857 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:35.813857 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:35.813857 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 172/337] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:35.819755 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c94049-5468-4943-abbb-9f25fecbf322 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:35.934566 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c94049-5468-4943-abbb-9f25fecbf322 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:35.975187 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856ef1c-d3fe-4f50-ba56-c18527734f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:35.975823 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856ef1c-d3fe-4f50-ba56-c18527734f28 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:35.976104 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856ef1c-d3fe-4f50-ba56-c18527734f28 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:35.976430 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856ef1c-d3fe-4f50-ba56-c18527734f28 None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:35.977112 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2856ef1c-d3fe-4f50-ba56-c18527734f28 None admin] RBAC: Adding query filter params (name=member) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:35.977436 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2856ef1c-d3fe-4f50-ba56-c18527734f28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:35.978905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2856ef1c-d3fe-4f50-ba56-c18527734f28 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:35.993513 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 166/338] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 10 22:34:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:36.000787 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a5df6-b4d2-4920-b09f-54cc7299dbdc None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:36.128195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:36.135272 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a5df6-b4d2-4920-b09f-54cc7299dbdc None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:36.175082 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e9677c-38d5-4216-9f85-afa1cce8c79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:36.175753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9677c-38d5-4216-9f85-afa1cce8c79b None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:36.176031 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9677c-38d5-4216-9f85-afa1cce8c79b None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:36.176300 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9677c-38d5-4216-9f85-afa1cce8c79b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:36.186970 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcc410-9abd-4b20-ad99-755c20549a3f None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:36.187272 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcc410-9abd-4b20-ad99-755c20549a3f None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:36.187564 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcc410-9abd-4b20-ad99-755c20549a3f None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:36.189219 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 167/339] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:36.195192 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf97e8d-384e-479e-ae2a-21a1df181a8d None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:36.195468 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf97e8d-384e-479e-ae2a-21a1df181a8d None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:36.195792 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf97e8d-384e-479e-ae2a-21a1df181a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:36.217905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-dbf97e8d-384e-479e-ae2a-21a1df181a8d None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:36.219087 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-dbf97e8d-384e-479e-ae2a-21a1df181a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:36.253366 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:36.265900 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e9677c-38d5-4216-9f85-afa1cce8c79b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:36.269301 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e9677c-38d5-4216-9f85-afa1cce8c79b None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:36.295827 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 168/340] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:36.297114 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf080227-6739-4aa9-8b37-8fe231219a7f None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:36.297478 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf080227-6739-4aa9-8b37-8fe231219a7f None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:36.297761 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf080227-6739-4aa9-8b37-8fe231219a7f None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:36.299849 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 169/341] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:36.305437 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed83a98-925c-4b20-a584-315fce3e61da None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:36.305712 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed83a98-925c-4b20-a584-315fce3e61da None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:36.305989 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed83a98-925c-4b20-a584-315fce3e61da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:36.327322 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-7ed83a98-925c-4b20-a584-315fce3e61da None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:36.328442 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-7ed83a98-925c-4b20-a584-315fce3e61da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:36.399371 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:36.405871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 170/342] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:34:36.406797 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed83a98-925c-4b20-a584-315fce3e61da None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:36.413046 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 173/343] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:34:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:34:36.421202 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e9677c-38d5-4216-9f85-afa1cce8c79b None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:36.556364 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:36.560440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed83a98-925c-4b20-a584-315fce3e61da None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:36.597139 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e9677c-38d5-4216-9f85-afa1cce8c79b None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:36.602428 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f9f7e6-2a12-4b13-99ca-79407725c28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:36.603075 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9f7e6-2a12-4b13-99ca-79407725c28b None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:36.603362 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9f7e6-2a12-4b13-99ca-79407725c28b None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:36.603639 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9f7e6-2a12-4b13-99ca-79407725c28b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:36.614930 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f9f7e6-2a12-4b13-99ca-79407725c28b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:36.619866 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-12f9f7e6-2a12-4b13-99ca-79407725c28b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:34:36.619866 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:36.619866 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:36.619866 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:36.619866 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:36.619866 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:36.619866 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:36.619866 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:36.619866 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:36.619866 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:36.619866 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:36.619866 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:36.619866 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:36.619866 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:36.619866 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:36.619866 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:36.620746 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:36.620746 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:36.620746 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:36.620746 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:36.620746 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:36.620746 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:36.620746 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:36.620746 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:36.620746 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:36.620746 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:36.620746 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:36.620746 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:36.620746 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:36.620746 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:36.620746 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:36.620746 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:36.621542 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:36.621542 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:36.621542 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:36.621542 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:36.621542 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:36.621542 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:36.621542 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:36.621542 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:36.621542 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:36.621542 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:36.621542 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:34:36.621542 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:36.621542 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 171/344] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 10 22:34:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:36.622871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f9f7e6-2a12-4b13-99ca-79407725c28b None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:36.650980 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96856fca-9e55-4d39-b70b-69ff86808c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:36.651632 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96856fca-9e55-4d39-b70b-69ff86808c44 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:36.651909 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96856fca-9e55-4d39-b70b-69ff86808c44 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:36.652183 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96856fca-9e55-4d39-b70b-69ff86808c44 None admin] PATH_INFO: `/v3/projects/2694f62f168849c0a5cd44bffc841cef` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:36.655359 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96856fca-9e55-4d39-b70b-69ff86808c44 None admin] RBAC: Authorizing `identity:get_project(project_id=2694f62f168849c0a5cd44bffc841cef)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:36.656851 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96856fca-9e55-4d39-b70b-69ff86808c44 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:36.658847 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 174/345] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:36 2022] GET /identity/v3/projects/2694f62f168849c0a5cd44bffc841cef => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:36.660002 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c396cbb9-31d8-43ec-a84b-d5cf93a6f3c8 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:36.660280 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c396cbb9-31d8-43ec-a84b-d5cf93a6f3c8 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:36.660551 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c396cbb9-31d8-43ec-a84b-d5cf93a6f3c8 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:36.662025 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 175/346] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:36.662717 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c396cbb9-31d8-43ec-a84b-d5cf93a6f3c8 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:36.736753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:36.737631 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f9f7e6-2a12-4b13-99ca-79407725c28b None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:36.781243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c396cbb9-31d8-43ec-a84b-d5cf93a6f3c8 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:36.792104 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8234e46-7387-4df3-8fa3-fbfb5a074890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:36.792856 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8234e46-7387-4df3-8fa3-fbfb5a074890 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:36.793141 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8234e46-7387-4df3-8fa3-fbfb5a074890 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:36.793470 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8234e46-7387-4df3-8fa3-fbfb5a074890 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:36.805508 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8234e46-7387-4df3-8fa3-fbfb5a074890 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:36.811710 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-d8234e46-7387-4df3-8fa3-fbfb5a074890 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:36.811710 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:36.811710 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:36.811710 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:36.811710 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:36.811710 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:36.811710 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:36.811710 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:36.811710 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:36.811710 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:36.811710 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:36.811710 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:36.811710 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:36.811710 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:36.811710 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:36.811710 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:36.813148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:36.813148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:36.813148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:36.813148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:36.813148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:36.813148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:36.813148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:36.813148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:36.813148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:36.813148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:36.813148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:36.813148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:36.813148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:36.813148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:36.813148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:36.813148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:36.814170 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:36.814170 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:36.814170 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:36.814170 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:36.814170 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:36.814170 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:36.814170 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:36.814170 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:36.814170 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:36.814170 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:36.814170 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:36.814170 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:36.814766 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 172/347] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:36.816128 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8234e46-7387-4df3-8fa3-fbfb5a074890 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:36.818543 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08475aa8-42a3-42d9-82dc-1b9fa0f64fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:36.819232 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08475aa8-42a3-42d9-82dc-1b9fa0f64fbe None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:36.819604 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08475aa8-42a3-42d9-82dc-1b9fa0f64fbe None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:36.819965 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08475aa8-42a3-42d9-82dc-1b9fa0f64fbe None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:36.820693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08475aa8-42a3-42d9-82dc-1b9fa0f64fbe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:36.821057 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08475aa8-42a3-42d9-82dc-1b9fa0f64fbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:36.822521 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08475aa8-42a3-42d9-82dc-1b9fa0f64fbe None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:36.837630 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 176/348] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:34:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:36.839054 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930bb86-5586-4e50-af4c-40f8e2039c5e None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:36.839418 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930bb86-5586-4e50-af4c-40f8e2039c5e None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:36.839755 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930bb86-5586-4e50-af4c-40f8e2039c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:36.847978 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:36.858861 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-f930bb86-5586-4e50-af4c-40f8e2039c5e None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:36.860099 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-f930bb86-5586-4e50-af4c-40f8e2039c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:36.931841 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 177/349] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:36.932690 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f930bb86-5586-4e50-af4c-40f8e2039c5e None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:36.940251 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8234e46-7387-4df3-8fa3-fbfb5a074890 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:36.978440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791a703-9b09-4bc4-9aff-89bfd3a62fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:36.979098 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791a703-9b09-4bc4-9aff-89bfd3a62fba None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:36.979373 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791a703-9b09-4bc4-9aff-89bfd3a62fba None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:36.979649 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791a703-9b09-4bc4-9aff-89bfd3a62fba None admin] PATH_INFO: `/v3/groups/5db23cc9336e463a999bb63286f5587e` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:36.986965 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9791a703-9b09-4bc4-9aff-89bfd3a62fba None admin] RBAC: Authorizing `identity:get_group(group_id=5db23cc9336e463a999bb63286f5587e)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:36.988448 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9791a703-9b09-4bc4-9aff-89bfd3a62fba None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:36.990454 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 173/350] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 10 22:34:36 2022] GET /identity/v3/groups/5db23cc9336e463a999bb63286f5587e => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:36.991315 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791a703-9b09-4bc4-9aff-89bfd3a62fba None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:37.019024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:37.065981 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f930bb86-5586-4e50-af4c-40f8e2039c5e None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:37.105591 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:37.106449 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc53a6-2495-455b-823f-44941bd1e3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:37.107050 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc53a6-2495-455b-823f-44941bd1e3a9 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:37.107166 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc53a6-2495-455b-823f-44941bd1e3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:37.107855 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc53a6-2495-455b-823f-44941bd1e3a9 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:37.109692 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dc53a6-2495-455b-823f-44941bd1e3a9 None admin] RBAC: Adding query filter params (name=member) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:37.110036 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dc53a6-2495-455b-823f-44941bd1e3a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:37.111571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dc53a6-2495-455b-823f-44941bd1e3a9 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:37.127702 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 178/351] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 10 22:34:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:37.128411 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc53a6-2495-455b-823f-44941bd1e3a9 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:37.141888 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791a703-9b09-4bc4-9aff-89bfd3a62fba None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:37.181022 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d32f1-799d-4bbb-9408-21595861e34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:37.182058 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d32f1-799d-4bbb-9408-21595861e34e None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:37.182594 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d32f1-799d-4bbb-9408-21595861e34e None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:37.183068 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d32f1-799d-4bbb-9408-21595861e34e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:37.243526 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc53a6-2495-455b-823f-44941bd1e3a9 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:37.260943 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:37.266933 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d32f1-799d-4bbb-9408-21595861e34e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:37.268729 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d32f1-799d-4bbb-9408-21595861e34e None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:37.281724 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17b70c3-0fa1-44c3-8851-5df69e8d0aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:37.282344 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17b70c3-0fa1-44c3-8851-5df69e8d0aa1 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:37.282617 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17b70c3-0fa1-44c3-8851-5df69e8d0aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:37.282883 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17b70c3-0fa1-44c3-8851-5df69e8d0aa1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:37.283736 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17b70c3-0fa1-44c3-8851-5df69e8d0aa1 None admin] RBAC: Adding query filter params (group.id=5db23cc9336e463a999bb63286f5587e, role.id=1085473c6e534f87812b05c7014f77c2, scope.project.id=2694f62f168849c0a5cd44bffc841cef) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:37.283918 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17b70c3-0fa1-44c3-8851-5df69e8d0aa1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:37.285338 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17b70c3-0fa1-44c3-8851-5df69e8d0aa1 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:37.291038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 179/352] 10.222.0.76 () {56 vars in 1376 bytes} [Wed Aug 10 22:34:37 2022] GET /identity/v3/role_assignments?group.id=5db23cc9336e463a999bb63286f5587e&scope.project.id=2694f62f168849c0a5cd44bffc841cef&role.id=1085473c6e534f87812b05c7014f77c2 => generated 260 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:37.291940 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17b70c3-0fa1-44c3-8851-5df69e8d0aa1 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:37.368946 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:37.397030 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 174/353] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:34:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:34:37.398244 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d32f1-799d-4bbb-9408-21595861e34e None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:37.412030 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17b70c3-0fa1-44c3-8851-5df69e8d0aa1 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:37.450112 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544cd9b6-97cb-4b1a-a6ee-a978dc0a01a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:37.450759 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cd9b6-97cb-4b1a-a6ee-a978dc0a01a3 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:37.451106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cd9b6-97cb-4b1a-a6ee-a978dc0a01a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:37.451493 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cd9b6-97cb-4b1a-a6ee-a978dc0a01a3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:37.463601 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544cd9b6-97cb-4b1a-a6ee-a978dc0a01a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:37.469769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-544cd9b6-97cb-4b1a-a6ee-a978dc0a01a3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:34:37.469769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:37.469769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:37.469769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:37.469769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:37.469769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:37.469769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:37.469769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:37.469769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:37.469769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:37.469769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:37.469769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:37.469769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:37.469769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:37.469769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:37.469769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:37.470744 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:37.470744 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:37.470744 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:37.470744 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:37.470744 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:37.470744 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:37.470744 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:37.470744 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:37.470744 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:37.470744 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:37.470744 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:37.470744 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:37.470744 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:37.470744 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:37.470744 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:37.470744 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:37.471528 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:37.471528 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:37.471528 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:37.471528 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:37.471528 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:37.471528 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:37.471528 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:37.471528 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:37.471528 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:37.471528 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:37.471528 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 22:34:37.471528 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:37.471528 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 180/354] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:37.472201 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544cd9b6-97cb-4b1a-a6ee-a978dc0a01a3 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:37.525055 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d32f1-799d-4bbb-9408-21595861e34e None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:37.544633 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:37.569135 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be505a30-0203-4ea8-9ed8-23e62845f7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:37.570614 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be505a30-0203-4ea8-9ed8-23e62845f7d2 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:37.571216 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be505a30-0203-4ea8-9ed8-23e62845f7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:37.571692 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be505a30-0203-4ea8-9ed8-23e62845f7d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:37.614889 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544cd9b6-97cb-4b1a-a6ee-a978dc0a01a3 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:37.633825 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:37.651618 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053f22f-a9da-4597-8fe6-2b27bdc4a2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:37.652305 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053f22f-a9da-4597-8fe6-2b27bdc4a2a1 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:37.652643 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053f22f-a9da-4597-8fe6-2b27bdc4a2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:37.652900 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053f22f-a9da-4597-8fe6-2b27bdc4a2a1 None admin] PATH_INFO: `/v3/projects/2694f62f168849c0a5cd44bffc841cef` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:37.653878 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be505a30-0203-4ea8-9ed8-23e62845f7d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:37.654184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4053f22f-a9da-4597-8fe6-2b27bdc4a2a1 None admin] RBAC: Authorizing `identity:get_project(project_id=2694f62f168849c0a5cd44bffc841cef)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:37.655769 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be505a30-0203-4ea8-9ed8-23e62845f7d2 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:37.656868 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4053f22f-a9da-4597-8fe6-2b27bdc4a2a1 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:37.658929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 181/355] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:37 2022] GET /identity/v3/projects/2694f62f168849c0a5cd44bffc841cef => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:37.662342 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053f22f-a9da-4597-8fe6-2b27bdc4a2a1 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:37.773582 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053f22f-a9da-4597-8fe6-2b27bdc4a2a1 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:37.785550 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:37.786804 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 175/356] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:34:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:34:37.790173 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a31c69-0010-4e56-b515-f0a3aa9416b9 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:37.790173 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a31c69-0010-4e56-b515-f0a3aa9416b9 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:37.790173 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a31c69-0010-4e56-b515-f0a3aa9416b9 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:37.790748 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 176/357] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:37.791505 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a31c69-0010-4e56-b515-f0a3aa9416b9 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:37.820474 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494ec18-52ae-4fcb-9e84-4b7b84714509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:37.821381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494ec18-52ae-4fcb-9e84-4b7b84714509 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:37.821983 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494ec18-52ae-4fcb-9e84-4b7b84714509 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:37.821983 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494ec18-52ae-4fcb-9e84-4b7b84714509 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:37.823316 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494ec18-52ae-4fcb-9e84-4b7b84714509 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:37.823316 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494ec18-52ae-4fcb-9e84-4b7b84714509 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:37.825426 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494ec18-52ae-4fcb-9e84-4b7b84714509 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:37.835595 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 182/358] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 10 22:34:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:37.836917 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e1d5e-7b73-4b4d-8c5a-80c0af234700 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:37.837323 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e1d5e-7b73-4b4d-8c5a-80c0af234700 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:37.837663 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e1d5e-7b73-4b4d-8c5a-80c0af234700 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:37.839131 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 183/359] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:37.840093 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e1d5e-7b73-4b4d-8c5a-80c0af234700 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:37.920822 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:37.944853 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a31c69-0010-4e56-b515-f0a3aa9416b9 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:37.974294 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e1d5e-7b73-4b4d-8c5a-80c0af234700 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:37.985882 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34da425-194a-494e-b650-bddca846e56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:37.986894 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34da425-194a-494e-b650-bddca846e56c None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:37.987314 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34da425-194a-494e-b650-bddca846e56c None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:37.987722 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34da425-194a-494e-b650-bddca846e56c None admin] PATH_INFO: `/v3/groups/5db23cc9336e463a999bb63286f5587e` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:37.989102 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34da425-194a-494e-b650-bddca846e56c None admin] RBAC: Authorizing `identity:get_group(group_id=5db23cc9336e463a999bb63286f5587e)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:37.990594 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34da425-194a-494e-b650-bddca846e56c None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:37.992454 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 177/360] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 10 22:34:37 2022] GET /identity/v3/groups/5db23cc9336e463a999bb63286f5587e => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:37.996981 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f605a45-dbb4-468d-a513-db79e64ab458 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:37.996981 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f605a45-dbb4-468d-a513-db79e64ab458 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:37.996981 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f605a45-dbb4-468d-a513-db79e64ab458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:38.016166 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-7f605a45-dbb4-468d-a513-db79e64ab458 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:38.018688 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-7f605a45-dbb4-468d-a513-db79e64ab458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:38.020226 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d66511-87b5-4642-977e-52fbb3c53143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:38.020836 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d66511-87b5-4642-977e-52fbb3c53143 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:38.021807 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d66511-87b5-4642-977e-52fbb3c53143 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:38.021807 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d66511-87b5-4642-977e-52fbb3c53143 None admin] PATH_INFO: `/v3/projects/d6cc740b2ba549809a3b7f863d43517c` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:38.022909 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d66511-87b5-4642-977e-52fbb3c53143 None admin] RBAC: Authorizing `identity:get_project(project_id=d6cc740b2ba549809a3b7f863d43517c)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:38.024971 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d66511-87b5-4642-977e-52fbb3c53143 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:38.027451 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 184/361] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:37 2022] GET /identity/v3/projects/d6cc740b2ba549809a3b7f863d43517c => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:38.028466 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d66511-87b5-4642-977e-52fbb3c53143 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:38.090141 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:38.105050 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 178/362] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:38.107071 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b38c4-98dd-4946-aa16-40abf6284d22 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:38.107667 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b38c4-98dd-4946-aa16-40abf6284d22 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:38.108205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b38c4-98dd-4946-aa16-40abf6284d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:38.134667 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-c12b38c4-98dd-4946-aa16-40abf6284d22 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:38.136630 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-c12b38c4-98dd-4946-aa16-40abf6284d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:38.187270 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d66511-87b5-4642-977e-52fbb3c53143 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:38.227086 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 179/363] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:38.228560 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b38c4-98dd-4946-aa16-40abf6284d22 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:38.232287 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16a149-3004-48cf-b931-db2af5f735fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:38.233833 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16a149-3004-48cf-b931-db2af5f735fc None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:38.234125 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16a149-3004-48cf-b931-db2af5f735fc None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:38.234396 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16a149-3004-48cf-b931-db2af5f735fc None admin] PATH_INFO: `/v3/users/62cfebaddb434256800d5f1f67673721` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:38.235557 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16a149-3004-48cf-b931-db2af5f735fc None admin] RBAC: Authorizing `identity:get_user(user_id=62cfebaddb434256800d5f1f67673721)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:38.236997 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16a149-3004-48cf-b931-db2af5f735fc None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:38.239187 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 185/364] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:34:38 2022] GET /identity/v3/users/62cfebaddb434256800d5f1f67673721 => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:38.240851 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16a149-3004-48cf-b931-db2af5f735fc None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:38.247633 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:38.358011 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b38c4-98dd-4946-aa16-40abf6284d22 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:38.375264 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16a149-3004-48cf-b931-db2af5f735fc None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:38.384219 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:38.409618 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213a731-bed0-482b-b553-0c85551a4b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:38.410413 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213a731-bed0-482b-b553-0c85551a4b12 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:38.410413 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213a731-bed0-482b-b553-0c85551a4b12 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:38.410568 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213a731-bed0-482b-b553-0c85551a4b12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:38.411912 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a213a731-bed0-482b-b553-0c85551a4b12 None admin] RBAC: Adding query filter params (group.id=5db23cc9336e463a999bb63286f5587e, role.id=e22e4d2b3d06480bbcfa4d2ee3387c90, scope.project.id=2694f62f168849c0a5cd44bffc841cef) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:38.411912 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a213a731-bed0-482b-b553-0c85551a4b12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:38.413361 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a213a731-bed0-482b-b553-0c85551a4b12 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:38.423123 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 180/365] 10.222.0.76 () {56 vars in 1376 bytes} [Wed Aug 10 22:34:38 2022] GET /identity/v3/role_assignments?group.id=5db23cc9336e463a999bb63286f5587e&scope.project.id=2694f62f168849c0a5cd44bffc841cef&role.id=e22e4d2b3d06480bbcfa4d2ee3387c90 => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:38.423123 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213a731-bed0-482b-b553-0c85551a4b12 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:38.450290 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1f816-b11f-4fef-a918-9cf8bda3e65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:38.450869 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1f816-b11f-4fef-a918-9cf8bda3e65f None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:38.451151 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1f816-b11f-4fef-a918-9cf8bda3e65f None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:38.451427 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1f816-b11f-4fef-a918-9cf8bda3e65f None admin] PATH_INFO: `/v3/groups/5db23cc9336e463a999bb63286f5587e` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:38.452834 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef1f816-b11f-4fef-a918-9cf8bda3e65f None admin] RBAC: Authorizing `identity:get_group(group_id=5db23cc9336e463a999bb63286f5587e)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:38.455363 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef1f816-b11f-4fef-a918-9cf8bda3e65f None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:38.457907 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 186/366] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 10 22:34:38 2022] GET /identity/v3/groups/5db23cc9336e463a999bb63286f5587e => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:38.458770 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1f816-b11f-4fef-a918-9cf8bda3e65f None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:38.523338 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:38.558369 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213a731-bed0-482b-b553-0c85551a4b12 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:38.599805 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75e5b4-286d-4516-8ec1-0ef5dfdd60a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:38.600734 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75e5b4-286d-4516-8ec1-0ef5dfdd60a8 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:38.601108 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75e5b4-286d-4516-8ec1-0ef5dfdd60a8 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:38.601511 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75e5b4-286d-4516-8ec1-0ef5dfdd60a8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:38.607318 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1f816-b11f-4fef-a918-9cf8bda3e65f None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:38.618595 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75e5b4-286d-4516-8ec1-0ef5dfdd60a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:38.628357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-5f75e5b4-286d-4516-8ec1-0ef5dfdd60a8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:34:38.628357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:38.628357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:38.628357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:38.628357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:38.628357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:38.628357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:38.628357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:38.628357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:38.628357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:38.628357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:38.628357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:38.628357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:38.628357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:38.628357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:38.628357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:38.629475 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:38.629475 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:38.629475 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:38.629475 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:38.629475 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:38.629475 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:38.629475 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:38.629475 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:38.629475 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:38.629475 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:38.629475 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:38.629475 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:38.629475 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:38.629475 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:38.629475 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:38.629475 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:38.630264 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:38.630264 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:38.630264 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:38.630264 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:38.630264 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:38.630264 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:38.630264 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:38.630264 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:38.630264 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:38.630264 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:38.630264 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:34:38.630264 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:38.630264 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 181/367] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 10 22:34:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:38.631275 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75e5b4-286d-4516-8ec1-0ef5dfdd60a8 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:38.679689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9800b-ca8d-46fe-96dd-f135ca49d562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:38.681151 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9800b-ca8d-46fe-96dd-f135ca49d562 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:38.682164 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9800b-ca8d-46fe-96dd-f135ca49d562 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:38.682448 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9800b-ca8d-46fe-96dd-f135ca49d562 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:38.683346 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee9800b-ca8d-46fe-96dd-f135ca49d562 None admin] RBAC: Adding query filter params (role.id=05a9201bdeee4f2ab18c648d5e6cb250, user.id=62cfebaddb434256800d5f1f67673721, scope.system=all) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:38.683804 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee9800b-ca8d-46fe-96dd-f135ca49d562 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:38.685569 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee9800b-ca8d-46fe-96dd-f135ca49d562 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:38.703188 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 187/368] 10.222.0.76 () {56 vars in 1308 bytes} [Wed Aug 10 22:34:38 2022] GET /identity/v3/role_assignments?user.id=62cfebaddb434256800d5f1f67673721&scope.system=all&role.id=05a9201bdeee4f2ab18c648d5e6cb250 => generated 510 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:38.704109 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9800b-ca8d-46fe-96dd-f135ca49d562 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:38.754862 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:38.850288 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75e5b4-286d-4516-8ec1-0ef5dfdd60a8 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:38.909682 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9800b-ca8d-46fe-96dd-f135ca49d562 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:38.910476 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a3cc12-2c45-491b-9c07-7fd8ed92f73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:38.911436 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3cc12-2c45-491b-9c07-7fd8ed92f73e None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:38.911715 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3cc12-2c45-491b-9c07-7fd8ed92f73e None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:38.911986 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3cc12-2c45-491b-9c07-7fd8ed92f73e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:38.928373 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a3cc12-2c45-491b-9c07-7fd8ed92f73e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:38.935414 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-d3a3cc12-2c45-491b-9c07-7fd8ed92f73e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:34:38.935414 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:38.935414 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:38.935414 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:38.935414 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:38.935414 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:38.935414 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:38.935414 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:38.935414 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:38.935414 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:38.935414 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:38.935414 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:38.935414 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:38.935414 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:38.935414 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:38.935414 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:38.936297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:38.936297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:38.936297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:38.936297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:38.936297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:38.936297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:38.936297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:38.936297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:38.936297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:38.936297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:38.936297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:38.936297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:38.936297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:38.936297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:38.936297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:38.936297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:38.937095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:38.937095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:38.937095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:38.937095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:38.937095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:38.937095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:38.937095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:38.937095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:38.937095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:38.937095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:38.937095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:34:38.937095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:38.937095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 182/369] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:34:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:38.937790 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a3cc12-2c45-491b-9c07-7fd8ed92f73e None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:38.938754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:38.957700 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29412a3-110d-4143-b0fb-2071658c59b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:38.959472 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29412a3-110d-4143-b0fb-2071658c59b7 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:38.959755 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29412a3-110d-4143-b0fb-2071658c59b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:38.960030 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29412a3-110d-4143-b0fb-2071658c59b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:38.961859 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29412a3-110d-4143-b0fb-2071658c59b7 None admin] RBAC: Adding query filter params (group.id=5db23cc9336e463a999bb63286f5587e, role.id=1085473c6e534f87812b05c7014f77c2, scope.project.id=d6cc740b2ba549809a3b7f863d43517c) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:38.962169 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29412a3-110d-4143-b0fb-2071658c59b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:38.963886 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29412a3-110d-4143-b0fb-2071658c59b7 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:38.977498 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 188/370] 10.222.0.76 () {56 vars in 1376 bytes} [Wed Aug 10 22:34:38 2022] GET /identity/v3/role_assignments?group.id=5db23cc9336e463a999bb63286f5587e&scope.project.id=d6cc740b2ba549809a3b7f863d43517c&role.id=1085473c6e534f87812b05c7014f77c2 => generated 260 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:38.978659 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29412a3-110d-4143-b0fb-2071658c59b7 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:39.082414 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a3cc12-2c45-491b-9c07-7fd8ed92f73e None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:39.119690 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:39.129068 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29412a3-110d-4143-b0fb-2071658c59b7 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:39.131395 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf1a19-5972-45a1-8cd6-d390707310a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:39.132225 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf1a19-5972-45a1-8cd6-d390707310a0 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:39.132591 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf1a19-5972-45a1-8cd6-d390707310a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:39.132902 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf1a19-5972-45a1-8cd6-d390707310a0 None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:39.133777 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf1a19-5972-45a1-8cd6-d390707310a0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:39.134314 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf1a19-5972-45a1-8cd6-d390707310a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:39.135721 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf1a19-5972-45a1-8cd6-d390707310a0 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:39.151754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 183/371] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:34:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:39.158391 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf1a19-5972-45a1-8cd6-d390707310a0 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:39.185885 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592eec09-4a57-41bf-abfb-95c1df6539ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:39.186499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592eec09-4a57-41bf-abfb-95c1df6539ff None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:39.186778 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592eec09-4a57-41bf-abfb-95c1df6539ff None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:39.187052 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592eec09-4a57-41bf-abfb-95c1df6539ff None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:39.187795 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592eec09-4a57-41bf-abfb-95c1df6539ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:39.188148 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592eec09-4a57-41bf-abfb-95c1df6539ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:39.189654 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592eec09-4a57-41bf-abfb-95c1df6539ff None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:39.202004 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 189/372] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:39.209393 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592eec09-4a57-41bf-abfb-95c1df6539ff None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:39.246041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:39.302158 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf1a19-5972-45a1-8cd6-d390707310a0 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:39.323125 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592eec09-4a57-41bf-abfb-95c1df6539ff None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:39.345205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40913526-47fc-4593-ac1b-2dd09e33ba67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:39.345866 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40913526-47fc-4593-ac1b-2dd09e33ba67 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:39.346150 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40913526-47fc-4593-ac1b-2dd09e33ba67 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:39.346423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40913526-47fc-4593-ac1b-2dd09e33ba67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:39.390006 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b285efff-c1c6-47e8-8a83-f6df35114cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:39.390636 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b285efff-c1c6-47e8-8a83-f6df35114cac None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:39.390914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b285efff-c1c6-47e8-8a83-f6df35114cac None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:39.391186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b285efff-c1c6-47e8-8a83-f6df35114cac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:39.445290 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40913526-47fc-4593-ac1b-2dd09e33ba67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:39.449882 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40913526-47fc-4593-ac1b-2dd09e33ba67 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:39.454264 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:39.527971 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b285efff-c1c6-47e8-8a83-f6df35114cac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:39.529802 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b285efff-c1c6-47e8-8a83-f6df35114cac None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:39.622377 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:39.645176 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 190/373] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:34:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:34:39.651573 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b285efff-c1c6-47e8-8a83-f6df35114cac None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:39.726003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 184/374] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:34:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 22:34:39.731809 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40913526-47fc-4593-ac1b-2dd09e33ba67 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:39.735789 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:39.814714 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b285efff-c1c6-47e8-8a83-f6df35114cac None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:39.872803 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a7f77-a412-4178-9dad-abad1535776c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:39.874092 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a7f77-a412-4178-9dad-abad1535776c None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:39.874713 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a7f77-a412-4178-9dad-abad1535776c None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:39.875078 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a7f77-a412-4178-9dad-abad1535776c None admin] PATH_INFO: `/v3/projects/946936eb2d5946a8b7f7ecfc0535ae77` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:39.876545 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a7f77-a412-4178-9dad-abad1535776c None admin] RBAC: Authorizing `identity:get_project(project_id=946936eb2d5946a8b7f7ecfc0535ae77)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:39.878421 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a7f77-a412-4178-9dad-abad1535776c None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:39.881487 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 191/375] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:39 2022] GET /identity/v3/projects/946936eb2d5946a8b7f7ecfc0535ae77 => generated 344 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:39.884224 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae6d8a-53be-4cc8-ad3b-38feec708557 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:39.884637 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae6d8a-53be-4cc8-ad3b-38feec708557 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:39.884917 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae6d8a-53be-4cc8-ad3b-38feec708557 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:39.887069 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 192/376] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:39.888216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae6d8a-53be-4cc8-ad3b-38feec708557 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:39.890969 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40913526-47fc-4593-ac1b-2dd09e33ba67 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:39.941079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aef323-0d63-42cd-8103-16d29232219a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:39.942490 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aef323-0d63-42cd-8103-16d29232219a None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:39.942773 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aef323-0d63-42cd-8103-16d29232219a None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:39.943047 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aef323-0d63-42cd-8103-16d29232219a None admin] PATH_INFO: `/v3/projects/d6cc740b2ba549809a3b7f863d43517c` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:39.944230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aef323-0d63-42cd-8103-16d29232219a None admin] RBAC: Authorizing `identity:get_project(project_id=d6cc740b2ba549809a3b7f863d43517c)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:39.945709 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aef323-0d63-42cd-8103-16d29232219a None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:39.948163 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 185/377] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:39 2022] GET /identity/v3/projects/d6cc740b2ba549809a3b7f863d43517c => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:34:39.949426 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba47f2-b78b-45df-b245-896ce9f8f2e3 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:39.950071 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba47f2-b78b-45df-b245-896ce9f8f2e3 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:39.950346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba47f2-b78b-45df-b245-896ce9f8f2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:39.972928 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-1dba47f2-b78b-45df-b245-896ce9f8f2e3 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:39.976003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-1dba47f2-b78b-45df-b245-896ce9f8f2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:39.999352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:40.041141 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae6d8a-53be-4cc8-ad3b-38feec708557 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:40.073509 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 186/378] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:40.074409 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba47f2-b78b-45df-b245-896ce9f8f2e3 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:40.081992 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:40.100616 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bcd2c2-5872-4987-906d-b2cd6c60fdac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:40.104237 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bcd2c2-5872-4987-906d-b2cd6c60fdac None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:40.104873 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bcd2c2-5872-4987-906d-b2cd6c60fdac None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:40.106356 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bcd2c2-5872-4987-906d-b2cd6c60fdac None admin] PATH_INFO: `/v3/groups/7a948e18bc7b4019850ab169882015ad` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:40.113453 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bcd2c2-5872-4987-906d-b2cd6c60fdac None admin] RBAC: Authorizing `identity:get_group(group_id=7a948e18bc7b4019850ab169882015ad)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:40.114926 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bcd2c2-5872-4987-906d-b2cd6c60fdac None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:40.116822 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 193/379] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 10 22:34:39 2022] GET /identity/v3/groups/7a948e18bc7b4019850ab169882015ad => generated 229 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:40.117805 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bcd2c2-5872-4987-906d-b2cd6c60fdac None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:40.202615 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba47f2-b78b-45df-b245-896ce9f8f2e3 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:40.230363 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bcd2c2-5872-4987-906d-b2cd6c60fdac None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:40.247197 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d9ea3-8ff8-40a3-9042-3493397297f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:40.248094 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d9ea3-8ff8-40a3-9042-3493397297f1 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:40.248378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d9ea3-8ff8-40a3-9042-3493397297f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:40.249305 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d9ea3-8ff8-40a3-9042-3493397297f1 None admin] PATH_INFO: `/v3/groups/5db23cc9336e463a999bb63286f5587e` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:40.250580 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8d9ea3-8ff8-40a3-9042-3493397297f1 None admin] RBAC: Authorizing `identity:get_group(group_id=5db23cc9336e463a999bb63286f5587e)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:40.253083 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8d9ea3-8ff8-40a3-9042-3493397297f1 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:40.255110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 187/380] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 10 22:34:40 2022] GET /identity/v3/groups/5db23cc9336e463a999bb63286f5587e => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:40.256092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d9ea3-8ff8-40a3-9042-3493397297f1 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:40.290049 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa393643-a733-492a-b818-95e4f545e6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:40.290686 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa393643-a733-492a-b818-95e4f545e6d2 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:40.291011 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa393643-a733-492a-b818-95e4f545e6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:40.291767 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa393643-a733-492a-b818-95e4f545e6d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:40.311133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa393643-a733-492a-b818-95e4f545e6d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:40.316280 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:40.323929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-fa393643-a733-492a-b818-95e4f545e6d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:40.323929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:40.323929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:40.323929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:40.323929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:40.323929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:40.323929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:40.323929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:40.323929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:40.323929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:40.323929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:40.323929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:40.323929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:40.323929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:40.323929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:40.323929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:40.324872 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:40.324872 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:40.324872 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:40.324872 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:40.324872 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:40.324872 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:40.324872 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:40.324872 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:40.324872 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:40.324872 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:40.324872 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:40.324872 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:40.324872 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:40.324872 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:40.324872 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:40.324872 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:40.325976 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:40.325976 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:40.325976 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:40.325976 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:40.325976 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:40.325976 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:40.325976 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:40.325976 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:40.325976 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:40.325976 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:40.325976 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:40.325976 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:40.330447 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 194/381] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:40.332534 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa393643-a733-492a-b818-95e4f545e6d2 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:40.405322 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:40.458659 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d9ea3-8ff8-40a3-9042-3493397297f1 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:40.476872 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa393643-a733-492a-b818-95e4f545e6d2 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:40.514986 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8098a4-faf1-454d-abeb-f1655fd4c48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:40.516025 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8098a4-faf1-454d-abeb-f1655fd4c48b None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:40.516722 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8098a4-faf1-454d-abeb-f1655fd4c48b None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:40.517151 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8098a4-faf1-454d-abeb-f1655fd4c48b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:40.518208 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8098a4-faf1-454d-abeb-f1655fd4c48b None admin] RBAC: Adding query filter params (group.id=7a948e18bc7b4019850ab169882015ad, role.id=6641e95fdc1f4c098ebc372bb692d227, scope.project.id=946936eb2d5946a8b7f7ecfc0535ae77) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:40.519959 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8098a4-faf1-454d-abeb-f1655fd4c48b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:40.521929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8098a4-faf1-454d-abeb-f1655fd4c48b None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:40.523556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23153a0-c6cb-4551-b37d-2ad5c00f4049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:40.524284 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23153a0-c6cb-4551-b37d-2ad5c00f4049 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:40.524781 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23153a0-c6cb-4551-b37d-2ad5c00f4049 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:40.525155 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23153a0-c6cb-4551-b37d-2ad5c00f4049 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:40.526028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23153a0-c6cb-4551-b37d-2ad5c00f4049 None admin] RBAC: Adding query filter params (group.id=5db23cc9336e463a999bb63286f5587e, role.id=e22e4d2b3d06480bbcfa4d2ee3387c90, scope.project.id=d6cc740b2ba549809a3b7f863d43517c) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:40.526866 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23153a0-c6cb-4551-b37d-2ad5c00f4049 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:40.528142 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23153a0-c6cb-4551-b37d-2ad5c00f4049 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:40.530472 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 188/382] 10.222.0.76 () {56 vars in 1376 bytes} [Wed Aug 10 22:34:40 2022] GET /identity/v3/role_assignments?group.id=7a948e18bc7b4019850ab169882015ad&scope.project.id=946936eb2d5946a8b7f7ecfc0535ae77&role.id=6641e95fdc1f4c098ebc372bb692d227 => generated 260 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:40.531878 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8098a4-faf1-454d-abeb-f1655fd4c48b None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:40.535664 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 195/383] 10.222.0.76 () {56 vars in 1376 bytes} [Wed Aug 10 22:34:40 2022] GET /identity/v3/role_assignments?group.id=5db23cc9336e463a999bb63286f5587e&scope.project.id=d6cc740b2ba549809a3b7f863d43517c&role.id=e22e4d2b3d06480bbcfa4d2ee3387c90 => generated 260 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:40.672619 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:40.680885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8098a4-faf1-454d-abeb-f1655fd4c48b None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:40.746928 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea116e-bcb2-4af6-bd44-f03205951442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:40.748022 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea116e-bcb2-4af6-bd44-f03205951442 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:40.748526 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea116e-bcb2-4af6-bd44-f03205951442 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:40.749006 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea116e-bcb2-4af6-bd44-f03205951442 None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:40.750177 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea116e-bcb2-4af6-bd44-f03205951442 None admin] RBAC: Adding query filter params (name=member) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:40.750965 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea116e-bcb2-4af6-bd44-f03205951442 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:40.752671 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea116e-bcb2-4af6-bd44-f03205951442 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:40.765851 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 189/384] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 10 22:34:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:40.771528 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23153a0-c6cb-4551-b37d-2ad5c00f4049 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:40.920873 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23153a0-c6cb-4551-b37d-2ad5c00f4049 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:40.949754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:40.964898 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23d230-621e-470d-b5b2-23cc8d828d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:40.965931 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23d230-621e-470d-b5b2-23cc8d828d8e None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:40.966431 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23d230-621e-470d-b5b2-23cc8d828d8e None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:40.966971 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23d230-621e-470d-b5b2-23cc8d828d8e None admin] PATH_INFO: `/v3/groups/5db23cc9336e463a999bb63286f5587e` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:40.968310 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae23d230-621e-470d-b5b2-23cc8d828d8e None admin] RBAC: Authorizing `identity:get_group(group_id=5db23cc9336e463a999bb63286f5587e)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:40.970059 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae23d230-621e-470d-b5b2-23cc8d828d8e None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:40.974059 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 196/385] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 10 22:34:40 2022] GET /identity/v3/groups/5db23cc9336e463a999bb63286f5587e => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:40.984341 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea116e-bcb2-4af6-bd44-f03205951442 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:41.149843 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea116e-bcb2-4af6-bd44-f03205951442 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:41.210180 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39537715-5a2a-438c-b06c-a0c63a3321c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:41.210761 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39537715-5a2a-438c-b06c-a0c63a3321c7 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:41.211045 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39537715-5a2a-438c-b06c-a0c63a3321c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:41.211326 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39537715-5a2a-438c-b06c-a0c63a3321c7 None admin] PATH_INFO: `/v3/projects/2694f62f168849c0a5cd44bffc841cef` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:41.212794 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39537715-5a2a-438c-b06c-a0c63a3321c7 None admin] RBAC: Authorizing `identity:get_project(project_id=2694f62f168849c0a5cd44bffc841cef)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:41.214276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39537715-5a2a-438c-b06c-a0c63a3321c7 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:41.216254 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 190/386] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:40 2022] GET /identity/v3/projects/2694f62f168849c0a5cd44bffc841cef => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:41.222608 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23d230-621e-470d-b5b2-23cc8d828d8e None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:41.297311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bae815-b77b-4119-a4e6-7a5c89e72f41 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:41.297743 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bae815-b77b-4119-a4e6-7a5c89e72f41 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:41.297992 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bae815-b77b-4119-a4e6-7a5c89e72f41 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:41.299343 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 191/387] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:41.304607 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761966f-b0a1-4811-bae4-eaaa0adc3d76 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:41.304894 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761966f-b0a1-4811-bae4-eaaa0adc3d76 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:41.305175 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761966f-b0a1-4811-bae4-eaaa0adc3d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:41.350024 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-8761966f-b0a1-4811-bae4-eaaa0adc3d76 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:41.351299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-8761966f-b0a1-4811-bae4-eaaa0adc3d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:41.353119 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23d230-621e-470d-b5b2-23cc8d828d8e None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:41.365653 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:41.393102 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c2354-a30b-43da-b089-ced34812a178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:41.393835 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c2354-a30b-43da-b089-ced34812a178 None admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:41.394114 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c2354-a30b-43da-b089-ced34812a178 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:41.394388 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c2354-a30b-43da-b089-ced34812a178 None admin] PATH_INFO: `/v3/projects/2694f62f168849c0a5cd44bffc841cef/groups/5db23cc9336e463a999bb63286f5587e/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:41.396320 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6c2354-a30b-43da-b089-ced34812a178 None admin] RBAC: Authorizing `identity:create_grant(project_id=2694f62f168849c0a5cd44bffc841cef, group_id=5db23cc9336e463a999bb63286f5587e, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:41.397805 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6c2354-a30b-43da-b089-ced34812a178 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:41.412686 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 197/388] 10.222.0.76 () {58 vars in 1442 bytes} [Wed Aug 10 22:34:41 2022] PUT /identity/v3/projects/2694f62f168849c0a5cd44bffc841cef/groups/5db23cc9336e463a999bb63286f5587e/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:34:41.536986 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 192/389] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:34:41.567120 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c2354-a30b-43da-b089-ced34812a178 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:41.641098 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:41.700462 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c2354-a30b-43da-b089-ced34812a178 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:41.741627 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203a3cb-d444-4e70-ba35-8309c7725b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:41.742270 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203a3cb-d444-4e70-ba35-8309c7725b99 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:41.742547 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203a3cb-d444-4e70-ba35-8309c7725b99 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:41.742818 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203a3cb-d444-4e70-ba35-8309c7725b99 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:41.754741 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b203a3cb-d444-4e70-ba35-8309c7725b99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:41.759920 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-b203a3cb-d444-4e70-ba35-8309c7725b99 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:34:41.759920 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:41.759920 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:41.759920 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:41.759920 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:41.759920 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:41.759920 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:41.759920 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:41.759920 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:41.759920 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:41.759920 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:41.759920 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:41.759920 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:41.759920 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:41.759920 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:41.759920 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:41.760808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:41.760808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:41.760808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:41.760808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:41.760808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:41.760808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:41.760808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:41.760808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:41.760808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:41.760808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:41.760808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:41.760808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:41.760808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:41.760808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:41.760808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:41.760808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:41.761777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:41.761777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:41.761777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:41.761777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:41.761777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:41.761777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:41.761777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:41.761777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:41.761777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:41.761777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:41.761777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:34:41.761777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:41.761777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 198/390] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 10 22:34:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:41.767570 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8761966f-b0a1-4811-bae4-eaaa0adc3d76 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:41.892256 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8761966f-b0a1-4811-bae4-eaaa0adc3d76 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:41.932432 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff5d3b0-99fc-48cb-97d2-9d75033ccdb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:41.933065 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5d3b0-99fc-48cb-97d2-9d75033ccdb1 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:41.933436 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5d3b0-99fc-48cb-97d2-9d75033ccdb1 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:41.933717 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5d3b0-99fc-48cb-97d2-9d75033ccdb1 None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:41.934391 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff5d3b0-99fc-48cb-97d2-9d75033ccdb1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:41.934694 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff5d3b0-99fc-48cb-97d2-9d75033ccdb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:41.936207 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff5d3b0-99fc-48cb-97d2-9d75033ccdb1 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:41.945754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 193/391] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:34:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:41.951043 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203a3cb-d444-4e70-ba35-8309c7725b99 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:42.053247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c61bd-c04e-40fc-991e-e0c3caa42536 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:42.053851 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c61bd-c04e-40fc-991e-e0c3caa42536 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:42.054321 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c61bd-c04e-40fc-991e-e0c3caa42536 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:42.055835 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 194/392] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:42.062124 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f94a54-50f2-480d-bb15-a8ad7fa74b2a None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:42.062976 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f94a54-50f2-480d-bb15-a8ad7fa74b2a None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:42.063528 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f94a54-50f2-480d-bb15-a8ad7fa74b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:42.071585 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203a3cb-d444-4e70-ba35-8309c7725b99 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:42.073614 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:42.090363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-97f94a54-50f2-480d-bb15-a8ad7fa74b2a None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:42.091933 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-97f94a54-50f2-480d-bb15-a8ad7fa74b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:42.098698 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:42.145147 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce4d03c-cfed-4efd-aa67-acf9ac31546d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:42.145753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4d03c-cfed-4efd-aa67-acf9ac31546d None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:42.146035 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4d03c-cfed-4efd-aa67-acf9ac31546d None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:42.146309 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4d03c-cfed-4efd-aa67-acf9ac31546d None admin] PATH_INFO: `/v3/groups/5db23cc9336e463a999bb63286f5587e` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:42.148872 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce4d03c-cfed-4efd-aa67-acf9ac31546d None admin] RBAC: Authorizing `identity:get_group(group_id=5db23cc9336e463a999bb63286f5587e)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:42.150376 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce4d03c-cfed-4efd-aa67-acf9ac31546d None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:42.154648 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 199/393] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 10 22:34:41 2022] GET /identity/v3/groups/5db23cc9336e463a999bb63286f5587e => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:42.160620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce4d03c-cfed-4efd-aa67-acf9ac31546d None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:42.260063 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 195/394] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:34:42.307481 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f94a54-50f2-480d-bb15-a8ad7fa74b2a None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:42.342964 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce4d03c-cfed-4efd-aa67-acf9ac31546d None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:42.406589 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845d55f-caba-40d6-bc28-6599346ed0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:42.407168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845d55f-caba-40d6-bc28-6599346ed0e8 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:42.407454 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845d55f-caba-40d6-bc28-6599346ed0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:42.407730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845d55f-caba-40d6-bc28-6599346ed0e8 None admin] PATH_INFO: `/v3/projects/2694f62f168849c0a5cd44bffc841cef` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:42.409047 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7845d55f-caba-40d6-bc28-6599346ed0e8 None admin] RBAC: Authorizing `identity:get_project(project_id=2694f62f168849c0a5cd44bffc841cef)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:42.410528 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7845d55f-caba-40d6-bc28-6599346ed0e8 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:42.412157 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 200/395] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:42 2022] GET /identity/v3/projects/2694f62f168849c0a5cd44bffc841cef => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:42.418488 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845d55f-caba-40d6-bc28-6599346ed0e8 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:42.489669 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:42.519296 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f94a54-50f2-480d-bb15-a8ad7fa74b2a None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:42.571374 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845d55f-caba-40d6-bc28-6599346ed0e8 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:42.612518 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb0c72-ef55-42a3-835a-e46534eb169a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:42.613199 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb0c72-ef55-42a3-835a-e46534eb169a None admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:42.613540 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb0c72-ef55-42a3-835a-e46534eb169a None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:42.613793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb0c72-ef55-42a3-835a-e46534eb169a None admin] PATH_INFO: `/v3/projects/2694f62f168849c0a5cd44bffc841cef/groups/5db23cc9336e463a999bb63286f5587e/roles/e22e4d2b3d06480bbcfa4d2ee3387c90` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:42.615995 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb0c72-ef55-42a3-835a-e46534eb169a None admin] RBAC: Authorizing `identity:create_grant(project_id=2694f62f168849c0a5cd44bffc841cef, group_id=5db23cc9336e463a999bb63286f5587e, role_id=e22e4d2b3d06480bbcfa4d2ee3387c90)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:42.617521 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb0c72-ef55-42a3-835a-e46534eb169a None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:42.632684 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 201/396] 10.222.0.76 () {58 vars in 1442 bytes} [Wed Aug 10 22:34:42 2022] PUT /identity/v3/projects/2694f62f168849c0a5cd44bffc841cef/groups/5db23cc9336e463a999bb63286f5587e/roles/e22e4d2b3d06480bbcfa4d2ee3387c90 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:34:42.634337 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41ae73-ba4f-4cb9-8968-4e56adf89f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:42.635161 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41ae73-ba4f-4cb9-8968-4e56adf89f4b None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:42.635709 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41ae73-ba4f-4cb9-8968-4e56adf89f4b None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:42.636102 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41ae73-ba4f-4cb9-8968-4e56adf89f4b None admin] PATH_INFO: `/v3/roles/member` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:42.649700 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:42.652616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c41ae73-ba4f-4cb9-8968-4e56adf89f4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:42.658367 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-6c41ae73-ba4f-4cb9-8968-4e56adf89f4b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:42.658367 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:42.658367 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:42.658367 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:42.658367 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:42.658367 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:42.658367 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:42.658367 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:42.658367 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:42.658367 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:42.658367 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:42.658367 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:42.658367 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:42.658367 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:42.658367 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:42.658367 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:42.659363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:42.659363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:42.659363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:42.659363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:42.659363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:42.659363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:42.659363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:42.659363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:42.659363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:42.659363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:42.659363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:42.659363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:42.659363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:42.659363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:42.659363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:42.659363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:42.660328 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:42.660328 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:42.660328 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:42.660328 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:42.660328 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:42.660328 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:42.660328 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:42.660328 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:42.660328 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:42.660328 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:42.660328 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:42.660328 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:42.660328 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 196/397] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:34:42.665255 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb0c72-ef55-42a3-835a-e46534eb169a None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:42.773829 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:42.782291 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb0c72-ef55-42a3-835a-e46534eb169a None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:42.829540 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26afc7ec-cb42-462f-a4da-919027e28df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:42.830167 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afc7ec-cb42-462f-a4da-919027e28df0 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:42.830450 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afc7ec-cb42-462f-a4da-919027e28df0 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:42.830726 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afc7ec-cb42-462f-a4da-919027e28df0 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:42.831406 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26afc7ec-cb42-462f-a4da-919027e28df0 None admin] RBAC: Adding query filter params (name=member) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:42.831710 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26afc7ec-cb42-462f-a4da-919027e28df0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:42.833087 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26afc7ec-cb42-462f-a4da-919027e28df0 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:42.843517 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 202/398] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 10 22:34:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:34:42.848875 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41ae73-ba4f-4cb9-8968-4e56adf89f4b None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:42.968495 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41ae73-ba4f-4cb9-8968-4e56adf89f4b None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:43.008956 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044675c-2e5b-4753-8981-8c25e22896b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:43.009658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044675c-2e5b-4753-8981-8c25e22896b9 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:43.009937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044675c-2e5b-4753-8981-8c25e22896b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:43.010209 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044675c-2e5b-4753-8981-8c25e22896b9 None admin] PATH_INFO: `/v3/groups/5db23cc9336e463a999bb63286f5587e` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:43.011832 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a044675c-2e5b-4753-8981-8c25e22896b9 None admin] RBAC: Authorizing `identity:get_group(group_id=5db23cc9336e463a999bb63286f5587e)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:43.013940 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a044675c-2e5b-4753-8981-8c25e22896b9 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:43.015836 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 197/399] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 10 22:34:42 2022] GET /identity/v3/groups/5db23cc9336e463a999bb63286f5587e => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:43.021853 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26afc7ec-cb42-462f-a4da-919027e28df0 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:43.193136 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26afc7ec-cb42-462f-a4da-919027e28df0 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:43.255499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27bbefd-e7f0-4a60-ba4b-984df58ddd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:43.256053 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27bbefd-e7f0-4a60-ba4b-984df58ddd5c None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:43.256332 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27bbefd-e7f0-4a60-ba4b-984df58ddd5c None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:43.256603 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27bbefd-e7f0-4a60-ba4b-984df58ddd5c None admin] PATH_INFO: `/v3/projects/d6cc740b2ba549809a3b7f863d43517c` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:43.257786 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27bbefd-e7f0-4a60-ba4b-984df58ddd5c None admin] RBAC: Authorizing `identity:get_project(project_id=d6cc740b2ba549809a3b7f863d43517c)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:43.259225 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27bbefd-e7f0-4a60-ba4b-984df58ddd5c None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:43.261067 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 203/400] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:43 2022] GET /identity/v3/projects/d6cc740b2ba549809a3b7f863d43517c => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:43.267295 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044675c-2e5b-4753-8981-8c25e22896b9 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:43.349745 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:43.379959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6600e-3c5f-4b49-b025-334bf36ca276 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:43.380598 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6600e-3c5f-4b49-b025-334bf36ca276 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:43.381077 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6600e-3c5f-4b49-b025-334bf36ca276 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:43.384329 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 204/401] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:34:43.389978 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249a1df-aa10-4b30-878d-0a976f3e4b52 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:43.390687 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249a1df-aa10-4b30-878d-0a976f3e4b52 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:43.391199 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249a1df-aa10-4b30-878d-0a976f3e4b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:43.404280 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044675c-2e5b-4753-8981-8c25e22896b9 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:43.414701 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-3249a1df-aa10-4b30-878d-0a976f3e4b52 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:43.416786 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-3249a1df-aa10-4b30-878d-0a976f3e4b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:43.446334 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:43.502043 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa6cfc-a271-492b-96e5-f6653f584607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:43.502745 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa6cfc-a271-492b-96e5-f6653f584607 None admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:43.503029 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa6cfc-a271-492b-96e5-f6653f584607 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:43.503308 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa6cfc-a271-492b-96e5-f6653f584607 None admin] PATH_INFO: `/v3/projects/d6cc740b2ba549809a3b7f863d43517c/groups/5db23cc9336e463a999bb63286f5587e/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:43.505339 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa6cfc-a271-492b-96e5-f6653f584607 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6cc740b2ba549809a3b7f863d43517c, group_id=5db23cc9336e463a999bb63286f5587e, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:43.506883 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa6cfc-a271-492b-96e5-f6653f584607 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:43.511682 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 205/402] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:34:43.538414 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249a1df-aa10-4b30-878d-0a976f3e4b52 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:43.544441 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 198/403] 10.222.0.76 () {58 vars in 1442 bytes} [Wed Aug 10 22:34:43 2022] PUT /identity/v3/projects/d6cc740b2ba549809a3b7f863d43517c/groups/5db23cc9336e463a999bb63286f5587e/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:34:43.599675 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e577b-f533-4689-bff7-29945dc9bfbb None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:43.599992 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e577b-f533-4689-bff7-29945dc9bfbb None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:43.600232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e577b-f533-4689-bff7-29945dc9bfbb None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:43.601866 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 199/404] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:43.607527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea4bb2-4639-4c51-a928-49892621eaf4 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:43.607811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea4bb2-4639-4c51-a928-49892621eaf4 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:43.608083 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea4bb2-4639-4c51-a928-49892621eaf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:43.662501 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-afea4bb2-4639-4c51-a928-49892621eaf4 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:43.663732 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-afea4bb2-4639-4c51-a928-49892621eaf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:43.680877 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249a1df-aa10-4b30-878d-0a976f3e4b52 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:43.730281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70785e6b-d172-4a28-b3d2-0192b80fbee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:43.731569 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70785e6b-d172-4a28-b3d2-0192b80fbee7 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:43.732056 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70785e6b-d172-4a28-b3d2-0192b80fbee7 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:43.733166 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70785e6b-d172-4a28-b3d2-0192b80fbee7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:43.749812 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70785e6b-d172-4a28-b3d2-0192b80fbee7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:43.756024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-70785e6b-d172-4a28-b3d2-0192b80fbee7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:34:43.756024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:43.756024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:43.756024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:43.756024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:43.756024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:43.756024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:43.756024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:43.756024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:43.756024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:43.756024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:43.756024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:43.756024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:43.756024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:43.756024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:43.756024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:43.757026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:43.757026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:43.757026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:43.757026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:43.757026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:43.757026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:43.757026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:43.757026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:43.757026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:43.757026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:43.757026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:43.757026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:43.757026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:43.757026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:43.757026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:43.757026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:43.757865 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:43.757865 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:43.757865 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:43.757865 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:43.757865 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:43.757865 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:43.757865 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:43.757865 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:43.757865 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:43.757865 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:43.757865 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:34:43.757865 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:43.762026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 206/405] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:34:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:43.767865 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70785e6b-d172-4a28-b3d2-0192b80fbee7 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:43.810435 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:43.830268 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 200/406] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:43.898379 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea4bb2-4639-4c51-a928-49892621eaf4 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:43.928894 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70785e6b-d172-4a28-b3d2-0192b80fbee7 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:43.982680 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4b0a3-c29a-416d-a766-237fa17ff28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:43.983316 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4b0a3-c29a-416d-a766-237fa17ff28c None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:43.983653 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4b0a3-c29a-416d-a766-237fa17ff28c None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:43.983931 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4b0a3-c29a-416d-a766-237fa17ff28c None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:43.984666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4b0a3-c29a-416d-a766-237fa17ff28c None admin] RBAC: Adding query filter params (name=admin) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:43.984972 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4b0a3-c29a-416d-a766-237fa17ff28c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:43.988309 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4b0a3-c29a-416d-a766-237fa17ff28c None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:44.008401 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 207/407] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:44.015470 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4b0a3-c29a-416d-a766-237fa17ff28c None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:44.031714 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:44.137030 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:44.160449 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4b0a3-c29a-416d-a766-237fa17ff28c None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:44.172711 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea4bb2-4639-4c51-a928-49892621eaf4 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:44.201737 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c485570-bff6-4e98-ad86-b5d3c69ba5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:44.202382 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c485570-bff6-4e98-ad86-b5d3c69ba5de None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:44.202870 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c485570-bff6-4e98-ad86-b5d3c69ba5de None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:44.203237 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c485570-bff6-4e98-ad86-b5d3c69ba5de None admin] PATH_INFO: `/v3/groups/7a948e18bc7b4019850ab169882015ad` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:44.205160 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c485570-bff6-4e98-ad86-b5d3c69ba5de None admin] RBAC: Authorizing `identity:get_group(group_id=7a948e18bc7b4019850ab169882015ad)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:44.210426 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c485570-bff6-4e98-ad86-b5d3c69ba5de None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:44.210426 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 208/408] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 10 22:34:44 2022] GET /identity/v3/groups/7a948e18bc7b4019850ab169882015ad => generated 229 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:34:44.215571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c485570-bff6-4e98-ad86-b5d3c69ba5de None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:44.243695 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e69d300-a069-4841-8718-a0591f52d759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:44.245164 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69d300-a069-4841-8718-a0591f52d759 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:44.245763 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69d300-a069-4841-8718-a0591f52d759 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:44.246378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69d300-a069-4841-8718-a0591f52d759 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:44.282345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e69d300-a069-4841-8718-a0591f52d759 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:44.287486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-4e69d300-a069-4841-8718-a0591f52d759 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:34:44.287486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:44.287486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:44.287486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:44.287486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:44.287486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:44.287486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:44.287486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:44.287486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:44.287486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:44.287486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:44.287486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:44.287486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:44.287486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:44.287486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:44.287486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:44.292914 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:44.292914 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:44.292914 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:44.292914 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:44.292914 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:44.292914 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:44.292914 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:44.292914 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:44.292914 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:44.292914 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:44.292914 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:44.292914 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:44.292914 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:44.292914 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:44.292914 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:44.292914 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:44.293739 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:44.293739 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:44.293739 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:44.293739 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:44.293739 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:44.293739 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:44.293739 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:44.293739 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:44.293739 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:44.293739 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:44.293739 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:34:44.293739 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:44.293739 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 201/409] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 10 22:34:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:44.295643 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7381a0-36ae-4852-9946-3124acbf30a7 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:44.298008 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7381a0-36ae-4852-9946-3124acbf30a7 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:44.298008 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7381a0-36ae-4852-9946-3124acbf30a7 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:44.299537 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 202/410] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:44.304579 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7381a0-36ae-4852-9946-3124acbf30a7 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:44.370291 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c485570-bff6-4e98-ad86-b5d3c69ba5de None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:44.427309 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e926a5f-34d6-4a42-a866-febf516ca948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:44.430815 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e926a5f-34d6-4a42-a866-febf516ca948 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:44.431356 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e926a5f-34d6-4a42-a866-febf516ca948 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:44.432371 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e926a5f-34d6-4a42-a866-febf516ca948 None admin] PATH_INFO: `/v3/projects/946936eb2d5946a8b7f7ecfc0535ae77` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:44.433931 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e926a5f-34d6-4a42-a866-febf516ca948 None admin] RBAC: Authorizing `identity:get_project(project_id=946936eb2d5946a8b7f7ecfc0535ae77)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:44.436108 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e926a5f-34d6-4a42-a866-febf516ca948 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:44.439239 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 209/411] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:44 2022] GET /identity/v3/projects/946936eb2d5946a8b7f7ecfc0535ae77 => generated 344 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:44.440846 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbac7a9-7b51-4cd3-91c7-e0904d8d8a06 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:44.453148 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbac7a9-7b51-4cd3-91c7-e0904d8d8a06 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:44.453148 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbac7a9-7b51-4cd3-91c7-e0904d8d8a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:44.460012 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7381a0-36ae-4852-9946-3124acbf30a7 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:44.470309 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-cdbac7a9-7b51-4cd3-91c7-e0904d8d8a06 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:44.471741 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-cdbac7a9-7b51-4cd3-91c7-e0904d8d8a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:44.489444 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:44.492701 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:44.524027 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e049f-33c4-48a1-a34f-353b4bc180e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:44.524969 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e049f-33c4-48a1-a34f-353b4bc180e8 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:44.526180 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e049f-33c4-48a1-a34f-353b4bc180e8 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:44.526646 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e049f-33c4-48a1-a34f-353b4bc180e8 None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:44.527788 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e049f-33c4-48a1-a34f-353b4bc180e8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:44.528282 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e049f-33c4-48a1-a34f-353b4bc180e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:44.530950 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e049f-33c4-48a1-a34f-353b4bc180e8 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:44.548805 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 203/412] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:34:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:44.550309 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e049f-33c4-48a1-a34f-353b4bc180e8 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:44.575821 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 210/413] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:44.577666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbac7a9-7b51-4cd3-91c7-e0904d8d8a06 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:44.672185 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e049f-33c4-48a1-a34f-353b4bc180e8 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:44.696497 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbac7a9-7b51-4cd3-91c7-e0904d8d8a06 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:44.707530 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f63eb-6f60-498b-a2fa-0b5d81cd7af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:44.708479 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f63eb-6f60-498b-a2fa-0b5d81cd7af4 None admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:44.708946 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f63eb-6f60-498b-a2fa-0b5d81cd7af4 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:44.709450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f63eb-6f60-498b-a2fa-0b5d81cd7af4 None admin] PATH_INFO: `/v3/projects/946936eb2d5946a8b7f7ecfc0535ae77/groups/7a948e18bc7b4019850ab169882015ad/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:44.711605 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f63eb-6f60-498b-a2fa-0b5d81cd7af4 None admin] RBAC: Authorizing `identity:create_grant(project_id=946936eb2d5946a8b7f7ecfc0535ae77, group_id=7a948e18bc7b4019850ab169882015ad, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:44.713927 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f63eb-6f60-498b-a2fa-0b5d81cd7af4 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:44.731375 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 204/414] 10.222.0.76 () {58 vars in 1442 bytes} [Wed Aug 10 22:34:44 2022] PUT /identity/v3/projects/946936eb2d5946a8b7f7ecfc0535ae77/groups/7a948e18bc7b4019850ab169882015ad/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:34:44.737522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c1465-de3b-4cba-829b-073457f300a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:44.737522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c1465-de3b-4cba-829b-073457f300a4 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:44.737522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c1465-de3b-4cba-829b-073457f300a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:44.737522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c1465-de3b-4cba-829b-073457f300a4 None admin] PATH_INFO: `/v3/groups/5db23cc9336e463a999bb63286f5587e` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:44.738095 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c1465-de3b-4cba-829b-073457f300a4 None admin] RBAC: Authorizing `identity:get_group(group_id=5db23cc9336e463a999bb63286f5587e)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:44.739491 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f63eb-6f60-498b-a2fa-0b5d81cd7af4 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:44.740857 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c1465-de3b-4cba-829b-073457f300a4 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:44.742741 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 211/415] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 10 22:34:44 2022] GET /identity/v3/groups/5db23cc9336e463a999bb63286f5587e => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:44.752820 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c1465-de3b-4cba-829b-073457f300a4 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:44.782961 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:44.791793 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:44.866297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f63eb-6f60-498b-a2fa-0b5d81cd7af4 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:44.907364 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d082c76-c7d6-4b68-a55d-4b7249e7149f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:44.908897 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d082c76-c7d6-4b68-a55d-4b7249e7149f None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:44.909411 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d082c76-c7d6-4b68-a55d-4b7249e7149f None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:44.909840 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d082c76-c7d6-4b68-a55d-4b7249e7149f None admin] PATH_INFO: `/v3/roles/member` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:44.924786 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d082c76-c7d6-4b68-a55d-4b7249e7149f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:44.930871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-4d082c76-c7d6-4b68-a55d-4b7249e7149f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:44.930871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:44.930871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:44.930871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:44.930871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:44.930871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:44.930871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:44.930871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:44.930871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:44.930871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:44.930871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:44.930871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:44.930871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:44.930871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:44.930871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:44.930871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:44.932092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:44.932092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:44.932092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:44.932092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:44.932092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:44.932092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:44.932092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:44.932092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:44.932092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:44.932092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:44.932092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:44.932092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:44.932092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:44.932092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:44.932092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:44.932092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:44.933081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:44.933081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:44.933081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:44.933081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:44.933081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:44.933081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:44.933081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:44.933081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:44.933081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:44.933081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:44.933081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:44.933081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:44.935607 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 205/416] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:44.941156 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d082c76-c7d6-4b68-a55d-4b7249e7149f None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:44.981704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c1465-de3b-4cba-829b-073457f300a4 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:45.077026 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:45.077841 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2150e1c-3796-4411-97c5-111db6d6382e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:45.078446 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2150e1c-3796-4411-97c5-111db6d6382e None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:45.078927 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2150e1c-3796-4411-97c5-111db6d6382e None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:45.079795 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2150e1c-3796-4411-97c5-111db6d6382e None admin] PATH_INFO: `/v3/projects/d6cc740b2ba549809a3b7f863d43517c` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:45.080903 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2150e1c-3796-4411-97c5-111db6d6382e None admin] RBAC: Authorizing `identity:get_project(project_id=d6cc740b2ba549809a3b7f863d43517c)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:45.082672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2150e1c-3796-4411-97c5-111db6d6382e None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:45.083257 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d082c76-c7d6-4b68-a55d-4b7249e7149f None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:45.085202 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 212/417] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:44 2022] GET /identity/v3/projects/d6cc740b2ba549809a3b7f863d43517c => generated 302 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:45.094295 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2150e1c-3796-4411-97c5-111db6d6382e None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:45.135579 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f0de71-b4e8-4d0c-802e-eea96fa69bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:45.136424 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0de71-b4e8-4d0c-802e-eea96fa69bf5 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:45.137150 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0de71-b4e8-4d0c-802e-eea96fa69bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:45.137600 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0de71-b4e8-4d0c-802e-eea96fa69bf5 None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:45.138764 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f0de71-b4e8-4d0c-802e-eea96fa69bf5 None admin] RBAC: Adding query filter params (name=member) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:45.139247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f0de71-b4e8-4d0c-802e-eea96fa69bf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:45.141501 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f0de71-b4e8-4d0c-802e-eea96fa69bf5 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:45.154155 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 206/418] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 10 22:34:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:45.160546 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f0de71-b4e8-4d0c-802e-eea96fa69bf5 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:45.211039 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:45.250585 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2150e1c-3796-4411-97c5-111db6d6382e None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:45.319501 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f0de71-b4e8-4d0c-802e-eea96fa69bf5 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:45.321246 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc349dcc-4fab-43cc-950e-bafe2687c2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:45.322599 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc349dcc-4fab-43cc-950e-bafe2687c2c7 None admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:45.324894 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc349dcc-4fab-43cc-950e-bafe2687c2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:45.325416 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc349dcc-4fab-43cc-950e-bafe2687c2c7 None admin] PATH_INFO: `/v3/projects/d6cc740b2ba549809a3b7f863d43517c/groups/5db23cc9336e463a999bb63286f5587e/roles/e22e4d2b3d06480bbcfa4d2ee3387c90` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:45.330009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc349dcc-4fab-43cc-950e-bafe2687c2c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6cc740b2ba549809a3b7f863d43517c, group_id=5db23cc9336e463a999bb63286f5587e, role_id=e22e4d2b3d06480bbcfa4d2ee3387c90)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:45.333445 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc349dcc-4fab-43cc-950e-bafe2687c2c7 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:45.361579 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 213/419] 10.222.0.76 () {58 vars in 1442 bytes} [Wed Aug 10 22:34:45 2022] PUT /identity/v3/projects/d6cc740b2ba549809a3b7f863d43517c/groups/5db23cc9336e463a999bb63286f5587e/roles/e22e4d2b3d06480bbcfa4d2ee3387c90 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:34:45.388884 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd19b75-46d5-412b-ae67-159c3e54d808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:45.389864 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd19b75-46d5-412b-ae67-159c3e54d808 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:45.390383 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd19b75-46d5-412b-ae67-159c3e54d808 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:45.390914 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd19b75-46d5-412b-ae67-159c3e54d808 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:45.420183 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71fef8-413b-4a3a-9cc6-ef644c75035c None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:45.420528 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71fef8-413b-4a3a-9cc6-ef644c75035c None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:45.420816 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71fef8-413b-4a3a-9cc6-ef644c75035c None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:45.422490 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 214/420] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:34:45.433310 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad265a5-0dea-4bbc-9099-803e7b96e005 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:45.433310 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad265a5-0dea-4bbc-9099-803e7b96e005 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:45.433310 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad265a5-0dea-4bbc-9099-803e7b96e005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:45.437320 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:45.495023 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd19b75-46d5-412b-ae67-159c3e54d808 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:45.496538 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd19b75-46d5-412b-ae67-159c3e54d808 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:45.502432 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-aad265a5-0dea-4bbc-9099-803e7b96e005 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:45.503642 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-aad265a5-0dea-4bbc-9099-803e7b96e005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:45.614433 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 207/421] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:34:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:34:45.620342 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd19b75-46d5-412b-ae67-159c3e54d808 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:45.672008 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 215/422] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:45.699692 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad265a5-0dea-4bbc-9099-803e7b96e005 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:45.714365 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:45.790170 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd19b75-46d5-412b-ae67-159c3e54d808 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:45.848735 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5748ce8-cd8a-4fdc-8165-a427cf22164f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:45.850828 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5748ce8-cd8a-4fdc-8165-a427cf22164f None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:45.852386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5748ce8-cd8a-4fdc-8165-a427cf22164f None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:45.853045 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5748ce8-cd8a-4fdc-8165-a427cf22164f None admin] PATH_INFO: `/v3/projects/2694f62f168849c0a5cd44bffc841cef` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:45.856435 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5748ce8-cd8a-4fdc-8165-a427cf22164f None admin] RBAC: Authorizing `identity:get_project(project_id=2694f62f168849c0a5cd44bffc841cef)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:45.856829 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:45.862488 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5748ce8-cd8a-4fdc-8165-a427cf22164f None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:45.866541 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 208/423] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:45 2022] GET /identity/v3/projects/2694f62f168849c0a5cd44bffc841cef => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:34:45.870387 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5748ce8-cd8a-4fdc-8165-a427cf22164f None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:45.929601 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad265a5-0dea-4bbc-9099-803e7b96e005 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:45.970121 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16dd702-5d3c-4b8d-ba99-dcec46bc55aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:45.970844 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16dd702-5d3c-4b8d-ba99-dcec46bc55aa None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:45.971218 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16dd702-5d3c-4b8d-ba99-dcec46bc55aa None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:45.971560 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16dd702-5d3c-4b8d-ba99-dcec46bc55aa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:45.983971 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16dd702-5d3c-4b8d-ba99-dcec46bc55aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:45.985895 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5748ce8-cd8a-4fdc-8165-a427cf22164f None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:45.989397 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-e16dd702-5d3c-4b8d-ba99-dcec46bc55aa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:34:45.989397 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:45.989397 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:45.989397 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:45.989397 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:45.989397 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:45.989397 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:45.989397 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:45.989397 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:45.989397 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:45.989397 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:45.989397 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:45.989397 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:45.989397 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:45.989397 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:45.989397 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:45.990342 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:45.990342 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:45.990342 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:45.990342 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:45.990342 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:45.990342 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:45.990342 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:45.990342 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:45.990342 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:45.990342 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:45.990342 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:45.990342 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:45.990342 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:45.990342 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:45.990342 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:45.990342 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:45.991132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:45.991132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:45.991132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:45.991132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:45.991132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:45.991132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:45.991132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:45.991132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:45.991132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:45.991132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:45.991132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:34:45.991132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:45.991132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 216/424] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 10 22:34:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:34:45.997432 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e676ac1-fc23-4f2c-b516-4ff1f289fe86 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:45.998204 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e676ac1-fc23-4f2c-b516-4ff1f289fe86 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:45.998204 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e676ac1-fc23-4f2c-b516-4ff1f289fe86 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:45.999838 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 217/425] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:45 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:46.000781 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e676ac1-fc23-4f2c-b516-4ff1f289fe86 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:46.018838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:46.027974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7eb799-84b4-4dd9-8cf2-46c428e67c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:46.028921 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7eb799-84b4-4dd9-8cf2-46c428e67c8a None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:46.029537 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7eb799-84b4-4dd9-8cf2-46c428e67c8a None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:46.030002 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7eb799-84b4-4dd9-8cf2-46c428e67c8a None admin] PATH_INFO: `/v3/groups/5db23cc9336e463a999bb63286f5587e` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:46.031384 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7eb799-84b4-4dd9-8cf2-46c428e67c8a None admin] RBAC: Authorizing `identity:get_group(group_id=5db23cc9336e463a999bb63286f5587e)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:46.033034 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7eb799-84b4-4dd9-8cf2-46c428e67c8a None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:46.035542 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 209/426] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 10 22:34:45 2022] GET /identity/v3/groups/5db23cc9336e463a999bb63286f5587e => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:34:46.036982 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f0ba9-eaa6-4f4a-9603-b7ae50b08095 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:46.037492 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f0ba9-eaa6-4f4a-9603-b7ae50b08095 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:46.038023 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f0ba9-eaa6-4f4a-9603-b7ae50b08095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:46.059067 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-8e5f0ba9-eaa6-4f4a-9603-b7ae50b08095 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:46.060525 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-8e5f0ba9-eaa6-4f4a-9603-b7ae50b08095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:46.127968 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e676ac1-fc23-4f2c-b516-4ff1f289fe86 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:46.138700 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 210/427] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:46.139860 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f0ba9-eaa6-4f4a-9603-b7ae50b08095 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:46.181247 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:46.191311 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad71c5-c1f5-49a5-8344-0bbd67352e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:46.192279 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad71c5-c1f5-49a5-8344-0bbd67352e54 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:46.192409 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad71c5-c1f5-49a5-8344-0bbd67352e54 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:46.192697 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad71c5-c1f5-49a5-8344-0bbd67352e54 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:46.195207 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad71c5-c1f5-49a5-8344-0bbd67352e54 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:46.195518 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad71c5-c1f5-49a5-8344-0bbd67352e54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:46.197118 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad71c5-c1f5-49a5-8344-0bbd67352e54 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:46.207512 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 218/428] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:34:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:46.208549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad71c5-c1f5-49a5-8344-0bbd67352e54 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:46.250782 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f0ba9-eaa6-4f4a-9603-b7ae50b08095 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:46.279591 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:46.290210 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e7af86-5f39-4c8b-a377-adf6745471e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:46.291197 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e7af86-5f39-4c8b-a377-adf6745471e4 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:46.291697 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e7af86-5f39-4c8b-a377-adf6745471e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:46.292142 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e7af86-5f39-4c8b-a377-adf6745471e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:46.293131 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e7af86-5f39-4c8b-a377-adf6745471e4 None admin] RBAC: Adding query filter params (group.id=5db23cc9336e463a999bb63286f5587e, role.id=1085473c6e534f87812b05c7014f77c2, scope.project.id=2694f62f168849c0a5cd44bffc841cef) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:46.293716 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e7af86-5f39-4c8b-a377-adf6745471e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:46.295540 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e7af86-5f39-4c8b-a377-adf6745471e4 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:46.303034 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 211/429] 10.222.0.76 () {56 vars in 1376 bytes} [Wed Aug 10 22:34:46 2022] GET /identity/v3/role_assignments?group.id=5db23cc9336e463a999bb63286f5587e&scope.project.id=2694f62f168849c0a5cd44bffc841cef&role.id=1085473c6e534f87812b05c7014f77c2 => generated 611 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:46.305104 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e7af86-5f39-4c8b-a377-adf6745471e4 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:46.325021 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad71c5-c1f5-49a5-8344-0bbd67352e54 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:46.367274 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796eaa5-9495-4cd9-8665-7b3299541705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:46.367971 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796eaa5-9495-4cd9-8665-7b3299541705 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:46.368255 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796eaa5-9495-4cd9-8665-7b3299541705 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:46.368529 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796eaa5-9495-4cd9-8665-7b3299541705 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:46.380524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e796eaa5-9495-4cd9-8665-7b3299541705 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:46.386384 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-e796eaa5-9495-4cd9-8665-7b3299541705 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:46.386384 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:46.386384 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:46.386384 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:46.386384 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:46.386384 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:46.386384 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:46.386384 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:46.386384 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:46.386384 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:46.386384 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:46.386384 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:46.386384 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:46.386384 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:46.386384 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:46.386384 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:46.387337 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:46.387337 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:46.387337 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:46.387337 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:46.387337 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:46.387337 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:46.387337 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:46.387337 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:46.387337 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:46.387337 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:46.387337 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:46.387337 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:46.387337 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:46.387337 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:46.387337 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:46.387337 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:46.388132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:46.388132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:46.388132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:46.388132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:46.388132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:46.388132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:46.388132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:46.388132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:46.388132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:46.388132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:46.388132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 22:34:46.388132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:46.392392 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 219/430] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:46.394289 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796eaa5-9495-4cd9-8665-7b3299541705 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:46.441139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e7af86-5f39-4c8b-a377-adf6745471e4 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:46.457699 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:46.485082 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c46bad-0a16-4eb5-9c7d-93d6ad1eafd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:46.486124 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c46bad-0a16-4eb5-9c7d-93d6ad1eafd9 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:46.486667 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c46bad-0a16-4eb5-9c7d-93d6ad1eafd9 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:46.487087 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c46bad-0a16-4eb5-9c7d-93d6ad1eafd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:46.509098 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796eaa5-9495-4cd9-8665-7b3299541705 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:46.548329 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17411cda-814f-4193-8f4c-53102a4cfbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:46.548624 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17411cda-814f-4193-8f4c-53102a4cfbac None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:46.548914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17411cda-814f-4193-8f4c-53102a4cfbac None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:46.549187 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17411cda-814f-4193-8f4c-53102a4cfbac None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:46.549933 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17411cda-814f-4193-8f4c-53102a4cfbac None admin] RBAC: Adding query filter params (name=member) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:46.550239 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17411cda-814f-4193-8f4c-53102a4cfbac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:46.551667 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17411cda-814f-4193-8f4c-53102a4cfbac None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:46.557094 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:46.567240 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 220/431] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 10 22:34:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:46.569591 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c46bad-0a16-4eb5-9c7d-93d6ad1eafd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:46.570542 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17411cda-814f-4193-8f4c-53102a4cfbac None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:46.571639 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c46bad-0a16-4eb5-9c7d-93d6ad1eafd9 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:46.683803 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17411cda-814f-4193-8f4c-53102a4cfbac None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:46.688397 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 212/432] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:34:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:34:46.694025 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c46bad-0a16-4eb5-9c7d-93d6ad1eafd9 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:46.701907 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:46.724136 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dd1326-8570-4d50-b628-659245f3c68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:46.724756 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd1326-8570-4d50-b628-659245f3c68b None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:46.725765 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd1326-8570-4d50-b628-659245f3c68b None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:46.726075 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd1326-8570-4d50-b628-659245f3c68b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:46.798294 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:46.806725 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dd1326-8570-4d50-b628-659245f3c68b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:46.808319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dd1326-8570-4d50-b628-659245f3c68b None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:46.810713 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c46bad-0a16-4eb5-9c7d-93d6ad1eafd9 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:46.849012 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb230a2-aaab-411a-b281-1bbcbef5779d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:46.850325 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb230a2-aaab-411a-b281-1bbcbef5779d None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:46.850825 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb230a2-aaab-411a-b281-1bbcbef5779d None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:46.851311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb230a2-aaab-411a-b281-1bbcbef5779d None admin] PATH_INFO: `/v3/projects/2694f62f168849c0a5cd44bffc841cef` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:46.852680 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb230a2-aaab-411a-b281-1bbcbef5779d None admin] RBAC: Authorizing `identity:get_project(project_id=2694f62f168849c0a5cd44bffc841cef)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:46.854336 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb230a2-aaab-411a-b281-1bbcbef5779d None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:46.856526 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 213/433] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:46 2022] GET /identity/v3/projects/2694f62f168849c0a5cd44bffc841cef => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:34:46.866703 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb230a2-aaab-411a-b281-1bbcbef5779d None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:46.928633 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 221/434] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:34:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:34:46.930049 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dd1326-8570-4d50-b628-659245f3c68b None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:46.949143 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:46.984968 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb230a2-aaab-411a-b281-1bbcbef5779d None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:47.028119 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37783004-94b4-4491-9291-a445a9cf7760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:47.029667 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37783004-94b4-4491-9291-a445a9cf7760 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:47.030151 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37783004-94b4-4491-9291-a445a9cf7760 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:47.030640 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37783004-94b4-4491-9291-a445a9cf7760 None admin] PATH_INFO: `/v3/groups/5db23cc9336e463a999bb63286f5587e` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:47.032028 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37783004-94b4-4491-9291-a445a9cf7760 None admin] RBAC: Authorizing `identity:get_group(group_id=5db23cc9336e463a999bb63286f5587e)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:47.033811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37783004-94b4-4491-9291-a445a9cf7760 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:47.035935 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 214/435] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 10 22:34:46 2022] GET /identity/v3/groups/5db23cc9336e463a999bb63286f5587e => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:47.044694 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37783004-94b4-4491-9291-a445a9cf7760 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:47.050434 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dd1326-8570-4d50-b628-659245f3c68b None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:47.078777 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:47.091006 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dcfe58-c993-45c9-b1d2-96571c3bd6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:47.091654 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dcfe58-c993-45c9-b1d2-96571c3bd6bf None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:47.091934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dcfe58-c993-45c9-b1d2-96571c3bd6bf None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:47.092211 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dcfe58-c993-45c9-b1d2-96571c3bd6bf None admin] PATH_INFO: `/v3/projects/d6cc740b2ba549809a3b7f863d43517c` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:47.093396 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dcfe58-c993-45c9-b1d2-96571c3bd6bf None admin] RBAC: Authorizing `identity:get_project(project_id=d6cc740b2ba549809a3b7f863d43517c)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:47.094845 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dcfe58-c993-45c9-b1d2-96571c3bd6bf None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:47.101154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 222/436] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:46 2022] GET /identity/v3/projects/d6cc740b2ba549809a3b7f863d43517c => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:47.103217 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dcfe58-c993-45c9-b1d2-96571c3bd6bf None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:47.173329 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37783004-94b4-4491-9291-a445a9cf7760 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:47.211212 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa8932-5ec0-4298-be12-c48c0bb09dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:47.211835 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa8932-5ec0-4298-be12-c48c0bb09dc6 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:47.212125 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa8932-5ec0-4298-be12-c48c0bb09dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:47.212406 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa8932-5ec0-4298-be12-c48c0bb09dc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:47.213346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa8932-5ec0-4298-be12-c48c0bb09dc6 None admin] RBAC: Adding query filter params (group.id=5db23cc9336e463a999bb63286f5587e, role.id=e22e4d2b3d06480bbcfa4d2ee3387c90, scope.project.id=2694f62f168849c0a5cd44bffc841cef) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:47.213668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa8932-5ec0-4298-be12-c48c0bb09dc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:47.215278 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa8932-5ec0-4298-be12-c48c0bb09dc6 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:47.222370 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 215/437] 10.222.0.76 () {56 vars in 1376 bytes} [Wed Aug 10 22:34:47 2022] GET /identity/v3/role_assignments?group.id=5db23cc9336e463a999bb63286f5587e&scope.project.id=2694f62f168849c0a5cd44bffc841cef&role.id=e22e4d2b3d06480bbcfa4d2ee3387c90 => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:47.241698 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:47.261145 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dcfe58-c993-45c9-b1d2-96571c3bd6bf None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:47.300958 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a3138e-d937-4184-89dc-3216e60da8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:47.301661 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3138e-d937-4184-89dc-3216e60da8ca None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:47.301966 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3138e-d937-4184-89dc-3216e60da8ca None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:47.302234 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3138e-d937-4184-89dc-3216e60da8ca None admin] PATH_INFO: `/v3/groups/5db23cc9336e463a999bb63286f5587e` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:47.303510 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a3138e-d937-4184-89dc-3216e60da8ca None admin] RBAC: Authorizing `identity:get_group(group_id=5db23cc9336e463a999bb63286f5587e)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:47.305081 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a3138e-d937-4184-89dc-3216e60da8ca None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:47.307078 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 223/438] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 10 22:34:47 2022] GET /identity/v3/groups/5db23cc9336e463a999bb63286f5587e => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:47.314543 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa8932-5ec0-4298-be12-c48c0bb09dc6 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:47.447276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:47.460766 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa8932-5ec0-4298-be12-c48c0bb09dc6 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:47.502210 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a35f22-c382-47bf-bfc7-6ad83512d5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:47.503245 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a35f22-c382-47bf-bfc7-6ad83512d5aa None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:47.503664 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a35f22-c382-47bf-bfc7-6ad83512d5aa None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:47.504267 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a35f22-c382-47bf-bfc7-6ad83512d5aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:47.506055 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a35f22-c382-47bf-bfc7-6ad83512d5aa None admin] RBAC: Adding query filter params (group.id=5db23cc9336e463a999bb63286f5587e, role.id=1085473c6e534f87812b05c7014f77c2, scope.project.id=d6cc740b2ba549809a3b7f863d43517c) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:47.506513 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a35f22-c382-47bf-bfc7-6ad83512d5aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:47.508122 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a35f22-c382-47bf-bfc7-6ad83512d5aa None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:47.515414 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 216/439] 10.222.0.76 () {56 vars in 1376 bytes} [Wed Aug 10 22:34:47 2022] GET /identity/v3/role_assignments?group.id=5db23cc9336e463a999bb63286f5587e&scope.project.id=d6cc740b2ba549809a3b7f863d43517c&role.id=1085473c6e534f87812b05c7014f77c2 => generated 611 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:47.527993 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5d89f-2439-46cb-812d-27196605958f None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:47.528521 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5d89f-2439-46cb-812d-27196605958f None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:47.528521 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5d89f-2439-46cb-812d-27196605958f None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:47.530247 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 224/440] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:34:47.536963 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2ab31-af8d-4ee1-8679-c893f078a22b None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:47.537733 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2ab31-af8d-4ee1-8679-c893f078a22b None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:47.538231 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2ab31-af8d-4ee1-8679-c893f078a22b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:47.561458 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-75b2ab31-af8d-4ee1-8679-c893f078a22b None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:47.563496 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-75b2ab31-af8d-4ee1-8679-c893f078a22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:47.654670 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 217/441] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:47.680671 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f5d89f-2439-46cb-812d-27196605958f None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:47.799650 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f5d89f-2439-46cb-812d-27196605958f None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:47.804537 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48674dd-11a4-4ec2-b627-cc4a7d87dcee None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:47.804925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48674dd-11a4-4ec2-b627-cc4a7d87dcee None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:47.805312 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48674dd-11a4-4ec2-b627-cc4a7d87dcee None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:47.806721 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 218/442] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:47.811909 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6acc2-b4d6-473c-ae35-3ee09b4b2ffa None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:47.812191 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6acc2-b4d6-473c-ae35-3ee09b4b2ffa None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:47.812465 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6acc2-b4d6-473c-ae35-3ee09b4b2ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:47.831963 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-a8f6acc2-b4d6-473c-ae35-3ee09b4b2ffa None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:47.833912 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-a8f6acc2-b4d6-473c-ae35-3ee09b4b2ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:47.839171 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aecb6a-a2d7-443f-9909-9575c5d816f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:47.839808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aecb6a-a2d7-443f-9909-9575c5d816f7 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:47.840085 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aecb6a-a2d7-443f-9909-9575c5d816f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:47.840357 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aecb6a-a2d7-443f-9909-9575c5d816f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:47.852386 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aecb6a-a2d7-443f-9909-9575c5d816f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:47.857106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-27aecb6a-a2d7-443f-9909-9575c5d816f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:34:47.857106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:47.857106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:47.857106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:47.857106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:47.857106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:47.857106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:47.857106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:47.857106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:47.857106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:47.857106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:47.857106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:47.857106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:47.857106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:47.857106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:47.857106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:47.858275 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:47.858275 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:47.858275 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:47.858275 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:47.858275 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:47.858275 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:47.858275 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:47.858275 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:47.858275 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:47.858275 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:47.858275 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:47.858275 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:47.858275 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:47.858275 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:47.858275 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:47.858275 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:47.859276 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:47.859276 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:47.859276 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:47.859276 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:47.859276 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:47.859276 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:47.859276 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:47.859276 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:47.859276 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:47.859276 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:47.859276 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:34:47.859276 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:47.859276 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 225/443] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:34:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:47.864937 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aecb6a-a2d7-443f-9909-9575c5d816f7 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:47.882182 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:47.917809 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 219/444] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:34:47.940622 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6acc2-b4d6-473c-ae35-3ee09b4b2ffa None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:47.959370 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:47.998545 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aecb6a-a2d7-443f-9909-9575c5d816f7 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:48.034149 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94fe8d7-f7c1-4bfb-a5f6-48c600a251ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:48.034757 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fe8d7-f7c1-4bfb-a5f6-48c600a251ee None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:48.035031 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fe8d7-f7c1-4bfb-a5f6-48c600a251ee None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:48.035303 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fe8d7-f7c1-4bfb-a5f6-48c600a251ee None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:48.036042 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94fe8d7-f7c1-4bfb-a5f6-48c600a251ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:48.036337 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94fe8d7-f7c1-4bfb-a5f6-48c600a251ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:48.037780 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94fe8d7-f7c1-4bfb-a5f6-48c600a251ee None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:48.045893 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 226/445] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:48.049853 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6acc2-b4d6-473c-ae35-3ee09b4b2ffa None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:48.051446 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94fe8d7-f7c1-4bfb-a5f6-48c600a251ee None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:48.085622 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0544012-742c-44c0-97ec-0488846f3a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:48.086260 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0544012-742c-44c0-97ec-0488846f3a87 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:48.086544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0544012-742c-44c0-97ec-0488846f3a87 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:48.086821 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0544012-742c-44c0-97ec-0488846f3a87 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:48.097078 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0544012-742c-44c0-97ec-0488846f3a87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:48.102127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-e0544012-742c-44c0-97ec-0488846f3a87 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:34:48.102127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:48.102127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:48.102127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:48.102127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:48.102127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:48.102127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:48.102127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:48.102127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:48.102127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:48.102127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:48.102127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:48.102127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:48.102127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:48.102127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:48.102127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:48.103118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:48.103118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:48.103118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:48.103118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:48.103118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:48.103118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:48.103118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:48.103118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:48.103118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:48.103118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:48.103118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:48.103118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:48.103118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:48.103118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:48.103118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:48.103118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:48.103979 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:48.103979 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:48.103979 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:48.103979 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:48.103979 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:48.103979 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:48.103979 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:48.103979 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:48.103979 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:48.103979 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:48.103979 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:34:48.103979 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:48.103979 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 220/446] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 10 22:34:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:34:48.109165 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0544012-742c-44c0-97ec-0488846f3a87 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:48.132728 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:48.159711 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94fe8d7-f7c1-4bfb-a5f6-48c600a251ee None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:48.186707 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:48.195394 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49839e60-426e-4135-88f4-bcc7de382c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:48.196002 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49839e60-426e-4135-88f4-bcc7de382c36 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:48.196278 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49839e60-426e-4135-88f4-bcc7de382c36 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:48.196542 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49839e60-426e-4135-88f4-bcc7de382c36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:48.214646 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0544012-742c-44c0-97ec-0488846f3a87 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:48.251798 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a49f0-3755-464d-91bf-83e02eb21eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:48.252407 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a49f0-3755-464d-91bf-83e02eb21eeb None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:48.252686 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a49f0-3755-464d-91bf-83e02eb21eeb None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:48.252966 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a49f0-3755-464d-91bf-83e02eb21eeb None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:48.253665 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a49f0-3755-464d-91bf-83e02eb21eeb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:48.254101 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a49f0-3755-464d-91bf-83e02eb21eeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:48.255589 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a49f0-3755-464d-91bf-83e02eb21eeb None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:48.263589 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 221/447] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:34:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:34:48.264749 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49839e60-426e-4135-88f4-bcc7de382c36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:48.266226 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49839e60-426e-4135-88f4-bcc7de382c36 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:48.273705 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a49f0-3755-464d-91bf-83e02eb21eeb None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:48.342765 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:48.370895 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 227/448] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:34:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 22:34:48.376411 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49839e60-426e-4135-88f4-bcc7de382c36 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:48.377061 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a49f0-3755-464d-91bf-83e02eb21eeb None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:48.412842 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ccafd7-6a91-42b4-ae5f-158e6458b879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:48.413483 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ccafd7-6a91-42b4-ae5f-158e6458b879 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:48.413766 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ccafd7-6a91-42b4-ae5f-158e6458b879 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:48.414040 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ccafd7-6a91-42b4-ae5f-158e6458b879 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:48.418700 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:48.481296 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49839e60-426e-4135-88f4-bcc7de382c36 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:48.487395 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ccafd7-6a91-42b4-ae5f-158e6458b879 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:48.489126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ccafd7-6a91-42b4-ae5f-158e6458b879 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:48.518269 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c25560-4ce5-474c-8d0f-31799835032f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:48.518913 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c25560-4ce5-474c-8d0f-31799835032f None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:48.519814 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c25560-4ce5-474c-8d0f-31799835032f None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:48.520219 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c25560-4ce5-474c-8d0f-31799835032f None admin] PATH_INFO: `/v3/projects/946936eb2d5946a8b7f7ecfc0535ae77` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:48.521705 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c25560-4ce5-474c-8d0f-31799835032f None admin] RBAC: Authorizing `identity:get_project(project_id=946936eb2d5946a8b7f7ecfc0535ae77)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:48.523156 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c25560-4ce5-474c-8d0f-31799835032f None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:48.524808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 228/449] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:48 2022] GET /identity/v3/projects/946936eb2d5946a8b7f7ecfc0535ae77 => generated 344 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:34:48.530210 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c25560-4ce5-474c-8d0f-31799835032f None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:48.575598 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:48.592117 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 222/450] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:34:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 22:34:48.598622 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ccafd7-6a91-42b4-ae5f-158e6458b879 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:48.632655 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:48.641082 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c25560-4ce5-474c-8d0f-31799835032f None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:48.675952 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4819dc7b-b3a4-4a48-b4b4-286722b58e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:48.676855 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819dc7b-b3a4-4a48-b4b4-286722b58e39 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:48.677132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819dc7b-b3a4-4a48-b4b4-286722b58e39 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:48.677449 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819dc7b-b3a4-4a48-b4b4-286722b58e39 None admin] PATH_INFO: `/v3/groups/7a948e18bc7b4019850ab169882015ad` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:48.678605 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4819dc7b-b3a4-4a48-b4b4-286722b58e39 None admin] RBAC: Authorizing `identity:get_group(group_id=7a948e18bc7b4019850ab169882015ad)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:48.680065 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4819dc7b-b3a4-4a48-b4b4-286722b58e39 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:48.681681 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 229/451] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 10 22:34:48 2022] GET /identity/v3/groups/7a948e18bc7b4019850ab169882015ad => generated 229 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:34:48.687380 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4819dc7b-b3a4-4a48-b4b4-286722b58e39 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:48.700687 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ccafd7-6a91-42b4-ae5f-158e6458b879 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:48.739435 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d74ea-b725-452f-af41-4198c0b2013e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:48.740074 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d74ea-b725-452f-af41-4198c0b2013e None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:48.740351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d74ea-b725-452f-af41-4198c0b2013e None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:48.740626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d74ea-b725-452f-af41-4198c0b2013e None admin] PATH_INFO: `/v3/projects/d6cc740b2ba549809a3b7f863d43517c` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:48.741804 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5d74ea-b725-452f-af41-4198c0b2013e None admin] RBAC: Authorizing `identity:get_project(project_id=d6cc740b2ba549809a3b7f863d43517c)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:48.743241 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5d74ea-b725-452f-af41-4198c0b2013e None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:48.745025 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 223/452] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:34:48 2022] GET /identity/v3/projects/d6cc740b2ba549809a3b7f863d43517c => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:34:48.750450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d74ea-b725-452f-af41-4198c0b2013e None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:48.790311 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4819dc7b-b3a4-4a48-b4b4-286722b58e39 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:48.802393 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:48.826337 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc9a000-eae7-4b52-8fcd-91a07d15b389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:48.826935 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9a000-eae7-4b52-8fcd-91a07d15b389 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:48.827212 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9a000-eae7-4b52-8fcd-91a07d15b389 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:48.827484 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9a000-eae7-4b52-8fcd-91a07d15b389 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:48.828195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc9a000-eae7-4b52-8fcd-91a07d15b389 None admin] RBAC: Adding query filter params (group.id=7a948e18bc7b4019850ab169882015ad, role.id=6641e95fdc1f4c098ebc372bb692d227, scope.project.id=946936eb2d5946a8b7f7ecfc0535ae77) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:48.828498 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc9a000-eae7-4b52-8fcd-91a07d15b389 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:48.829976 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc9a000-eae7-4b52-8fcd-91a07d15b389 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:48.836625 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 230/453] 10.222.0.76 () {56 vars in 1376 bytes} [Wed Aug 10 22:34:48 2022] GET /identity/v3/role_assignments?group.id=7a948e18bc7b4019850ab169882015ad&scope.project.id=946936eb2d5946a8b7f7ecfc0535ae77&role.id=6641e95fdc1f4c098ebc372bb692d227 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:34:48.859013 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d74ea-b725-452f-af41-4198c0b2013e None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:48.861894 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:48.897078 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43824c-284f-4a6d-8cf1-fb422e4319f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:48.897754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43824c-284f-4a6d-8cf1-fb422e4319f2 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:48.898036 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43824c-284f-4a6d-8cf1-fb422e4319f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:48.898311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43824c-284f-4a6d-8cf1-fb422e4319f2 None admin] PATH_INFO: `/v3/groups/5db23cc9336e463a999bb63286f5587e` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:48.899475 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43824c-284f-4a6d-8cf1-fb422e4319f2 None admin] RBAC: Authorizing `identity:get_group(group_id=5db23cc9336e463a999bb63286f5587e)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:48.900945 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43824c-284f-4a6d-8cf1-fb422e4319f2 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:48.902623 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 224/454] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 10 22:34:48 2022] GET /identity/v3/groups/5db23cc9336e463a999bb63286f5587e => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:48.908653 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc9a000-eae7-4b52-8fcd-91a07d15b389 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:49.020804 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc9a000-eae7-4b52-8fcd-91a07d15b389 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:49.056905 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f70157-57c4-47de-a8bc-eb278549b792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:49.057537 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f70157-57c4-47de-a8bc-eb278549b792 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:49.057812 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f70157-57c4-47de-a8bc-eb278549b792 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:49.058078 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f70157-57c4-47de-a8bc-eb278549b792 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:49.058791 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f70157-57c4-47de-a8bc-eb278549b792 None admin] RBAC: Adding query filter params (group.id=5db23cc9336e463a999bb63286f5587e, role.id=e22e4d2b3d06480bbcfa4d2ee3387c90, scope.project.id=d6cc740b2ba549809a3b7f863d43517c) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:49.059092 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f70157-57c4-47de-a8bc-eb278549b792 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:49.060507 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f70157-57c4-47de-a8bc-eb278549b792 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:49.067041 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 231/455] 10.222.0.76 () {56 vars in 1376 bytes} [Wed Aug 10 22:34:48 2022] GET /identity/v3/role_assignments?group.id=5db23cc9336e463a999bb63286f5587e&scope.project.id=d6cc740b2ba549809a3b7f863d43517c&role.id=e22e4d2b3d06480bbcfa4d2ee3387c90 => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:51.631988 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc80cf8-db41-41b2-834f-d64462218487 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:51.632526 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc80cf8-db41-41b2-834f-d64462218487 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:51.632526 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc80cf8-db41-41b2-834f-d64462218487 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:51.634128 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 225/456] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:51.639767 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c891fc-bf3f-4a49-82ec-b06748443a3a None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:51.640207 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c891fc-bf3f-4a49-82ec-b06748443a3a None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:51.640378 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c891fc-bf3f-4a49-82ec-b06748443a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:51.665443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-96c891fc-bf3f-4a49-82ec-b06748443a3a None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:51.666647 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-96c891fc-bf3f-4a49-82ec-b06748443a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:51.718474 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:51.773342 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 232/457] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:34:51.827040 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc80cf8-db41-41b2-834f-d64462218487 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:51.957999 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc80cf8-db41-41b2-834f-d64462218487 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:51.982968 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc6c75-ddd6-4c27-af8d-d58907d41107 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:51.983258 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc6c75-ddd6-4c27-af8d-d58907d41107 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:51.983441 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc6c75-ddd6-4c27-af8d-d58907d41107 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:51.985647 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 233/458] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:51.998700 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b74d3-4c10-433d-86bf-8386ebd82f8c None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:51.998991 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b74d3-4c10-433d-86bf-8386ebd82f8c None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:51.999240 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b74d3-4c10-433d-86bf-8386ebd82f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:52.004833 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756c1c1-ecac-4ec5-b1c4-e53c8618325e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:52.005501 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756c1c1-ecac-4ec5-b1c4-e53c8618325e None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:52.005780 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756c1c1-ecac-4ec5-b1c4-e53c8618325e None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:52.006055 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756c1c1-ecac-4ec5-b1c4-e53c8618325e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:52.026093 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5756c1c1-ecac-4ec5-b1c4-e53c8618325e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:52.027570 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5756c1c1-ecac-4ec5-b1c4-e53c8618325e None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:52.029037 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-d64b74d3-4c10-433d-86bf-8386ebd82f8c None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:52.030527 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-d64b74d3-4c10-433d-86bf-8386ebd82f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:52.050325 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-5756c1c1-ecac-4ec5-b1c4-e53c8618325e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:34:52.050325 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:52.050325 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:52.050325 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:52.050325 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:52.050325 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:52.050325 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:52.050325 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:52.050325 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:52.050325 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:52.050325 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:52.050325 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:52.050325 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:34:52.050325 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:34:52.050325 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:34:52.050325 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:34:52.051307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:52.051307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:52.051307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:52.051307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:52.051307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:52.051307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:52.051307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:52.051307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:52.051307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:52.051307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:52.051307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:52.051307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:52.051307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:52.051307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:52.051307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:52.051307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:52.051307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:34:52.052583 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:34:52.052583 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:34:52.052583 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:52.052583 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 226/459] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:34:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:34:52.057525 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756c1c1-ecac-4ec5-b1c4-e53c8618325e None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:52.117645 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:52.160829 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 234/460] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:52.185100 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b74d3-4c10-433d-86bf-8386ebd82f8c None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:52.253861 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756c1c1-ecac-4ec5-b1c4-e53c8618325e None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:52.290109 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe36819-76e9-48e2-b0ab-90f61a0a7e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:52.290960 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe36819-76e9-48e2-b0ab-90f61a0a7e12 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:52.291387 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe36819-76e9-48e2-b0ab-90f61a0a7e12 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:52.291809 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe36819-76e9-48e2-b0ab-90f61a0a7e12 None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:52.292622 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe36819-76e9-48e2-b0ab-90f61a0a7e12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:52.293108 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe36819-76e9-48e2-b0ab-90f61a0a7e12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:52.294757 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe36819-76e9-48e2-b0ab-90f61a0a7e12 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:52.298176 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:52.302892 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b74d3-4c10-433d-86bf-8386ebd82f8c None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:52.310206 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 227/461] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:34:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:34:52.316572 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab539bf-466b-4cfe-88ae-fb22f7789ca2 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:52.317232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab539bf-466b-4cfe-88ae-fb22f7789ca2 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:52.317742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab539bf-466b-4cfe-88ae-fb22f7789ca2 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:52.319492 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 228/462] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:52.320410 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab539bf-466b-4cfe-88ae-fb22f7789ca2 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:52.351456 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf91bde-0b8f-44d6-9db9-a6f725c3ae16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:52.352454 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf91bde-0b8f-44d6-9db9-a6f725c3ae16 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:52.353181 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf91bde-0b8f-44d6-9db9-a6f725c3ae16 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:52.353688 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf91bde-0b8f-44d6-9db9-a6f725c3ae16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:52.371342 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf91bde-0b8f-44d6-9db9-a6f725c3ae16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:52.373461 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf91bde-0b8f-44d6-9db9-a6f725c3ae16 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:52.396767 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-fdf91bde-0b8f-44d6-9db9-a6f725c3ae16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:34:52.396767 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:52.396767 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:52.396767 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:52.396767 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:52.396767 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:52.396767 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:52.396767 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:52.396767 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:52.396767 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:52.396767 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:52.396767 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:52.396767 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:34:52.396767 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:34:52.396767 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:34:52.396767 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:34:52.397790 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:52.397790 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:52.397790 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:52.397790 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:52.397790 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:52.397790 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:52.397790 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:52.397790 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:52.397790 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:52.397790 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:52.397790 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:52.397790 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:52.397790 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:52.397790 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:52.397790 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:52.397790 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:52.397790 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:34:52.398638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:34:52.398638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:34:52.398638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:52.401373 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 235/463] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:34:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:34:52.403193 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ea12b-bb9b-4125-83ea-6a5ddd5d5dd6 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:52.403697 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ea12b-bb9b-4125-83ea-6a5ddd5d5dd6 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:52.404416 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ea12b-bb9b-4125-83ea-6a5ddd5d5dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:52.428691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-d39ea12b-bb9b-4125-83ea-6a5ddd5d5dd6 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:52.430514 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-d39ea12b-bb9b-4125-83ea-6a5ddd5d5dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:52.439001 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:52.460944 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab539bf-466b-4cfe-88ae-fb22f7789ca2 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:52.501006 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f6edd6-a572-40e7-a495-4d0d46a8c626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:52.501742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6edd6-a572-40e7-a495-4d0d46a8c626 None admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:52.502027 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6edd6-a572-40e7-a495-4d0d46a8c626 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:52.502297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6edd6-a572-40e7-a495-4d0d46a8c626 None admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:52.502810 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 236/464] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:52.503667 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ea12b-bb9b-4125-83ea-6a5ddd5d5dd6 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:52.509800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f6edd6-a572-40e7-a495-4d0d46a8c626 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:52.511200 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f6edd6-a572-40e7-a495-4d0d46a8c626 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:52.542996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 229/465] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 10 22:34:52 2022] POST /identity/v3/users => generated 246 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:34:52.542996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727125aa-aa10-4ce5-9c98-375d47aa0e41 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:52.542996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727125aa-aa10-4ce5-9c98-375d47aa0e41 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:52.542996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727125aa-aa10-4ce5-9c98-375d47aa0e41 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:52.542996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 230/466] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:52.542996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727125aa-aa10-4ce5-9c98-375d47aa0e41 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:52.622503 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:52.646697 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ea12b-bb9b-4125-83ea-6a5ddd5d5dd6 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:52.686248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e831f2b-aff4-4ba1-a0ca-34c73a6d875d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:52.687059 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e831f2b-aff4-4ba1-a0ca-34c73a6d875d None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:52.687434 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e831f2b-aff4-4ba1-a0ca-34c73a6d875d None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:52.688349 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e831f2b-aff4-4ba1-a0ca-34c73a6d875d None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:52.689104 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e831f2b-aff4-4ba1-a0ca-34c73a6d875d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:52.689549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e831f2b-aff4-4ba1-a0ca-34c73a6d875d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:52.691126 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e831f2b-aff4-4ba1-a0ca-34c73a6d875d None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:52.692826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727125aa-aa10-4ce5-9c98-375d47aa0e41 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:52.712340 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 237/467] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:34:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:52.713777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7da11-170f-451b-8b5a-43f9cfd0a395 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:52.714065 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7da11-170f-451b-8b5a-43f9cfd0a395 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:52.714343 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7da11-170f-451b-8b5a-43f9cfd0a395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:52.734462 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-56a7da11-170f-451b-8b5a-43f9cfd0a395 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:52.736195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-56a7da11-170f-451b-8b5a-43f9cfd0a395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:52.741102 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e85648-8b2d-4d60-a9af-a40d63a1dbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:52.741771 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e85648-8b2d-4d60-a9af-a40d63a1dbed None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:52.742048 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e85648-8b2d-4d60-a9af-a40d63a1dbed None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:52.742314 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e85648-8b2d-4d60-a9af-a40d63a1dbed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:52.753171 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:52.758190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e85648-8b2d-4d60-a9af-a40d63a1dbed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:52.760872 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e85648-8b2d-4d60-a9af-a40d63a1dbed None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:52.779947 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-82e85648-8b2d-4d60-a9af-a40d63a1dbed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:34:52.779947 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:52.779947 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:52.779947 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:52.779947 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:52.779947 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:52.779947 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:52.779947 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:52.779947 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:52.779947 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:52.779947 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:52.779947 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:52.779947 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:34:52.779947 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:34:52.779947 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:34:52.779947 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:34:52.780896 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:52.780896 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:52.780896 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:52.780896 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:52.780896 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:52.780896 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:52.780896 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:52.780896 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:52.780896 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:52.780896 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:52.780896 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:52.780896 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:52.780896 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:52.780896 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:52.780896 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:52.780896 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:52.780896 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:34:52.782387 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:34:52.782387 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:34:52.782387 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:52.786340 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 231/468] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:34:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:52.789246 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e85648-8b2d-4d60-a9af-a40d63a1dbed None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:52.841430 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 238/469] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:52.842328 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7da11-170f-451b-8b5a-43f9cfd0a395 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:52.923984 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e85648-8b2d-4d60-a9af-a40d63a1dbed None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:52.944619 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:52.966421 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108b059-f848-4d09-87b1-4fdca0609e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:52.967252 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108b059-f848-4d09-87b1-4fdca0609e56 None admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:52.967607 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108b059-f848-4d09-87b1-4fdca0609e56 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:52.967952 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108b059-f848-4d09-87b1-4fdca0609e56 None admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:52.968661 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6108b059-f848-4d09-87b1-4fdca0609e56 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:52.970284 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6108b059-f848-4d09-87b1-4fdca0609e56 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:52.987234 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7da11-170f-451b-8b5a-43f9cfd0a395 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:52.997832 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 232/470] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 10 22:34:52 2022] POST /identity/v3/users => generated 248 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:34:52.999113 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108b059-f848-4d09-87b1-4fdca0609e56 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:53.028060 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22680af6-2acb-47bd-9435-042c1009d95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:53.028710 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22680af6-2acb-47bd-9435-042c1009d95e None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:53.029144 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22680af6-2acb-47bd-9435-042c1009d95e None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:53.029422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22680af6-2acb-47bd-9435-042c1009d95e None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:53.030164 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22680af6-2acb-47bd-9435-042c1009d95e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:53.030567 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22680af6-2acb-47bd-9435-042c1009d95e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:53.033883 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22680af6-2acb-47bd-9435-042c1009d95e None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:53.047649 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 239/471] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:34:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:53.053693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22680af6-2acb-47bd-9435-042c1009d95e None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:53.067677 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:53.134454 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108b059-f848-4d09-87b1-4fdca0609e56 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:53.181610 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a9124e-3437-4309-aa32-3d2c974c315a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:53.181814 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22680af6-2acb-47bd-9435-042c1009d95e None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:53.182746 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a9124e-3437-4309-aa32-3d2c974c315a None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:53.183254 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a9124e-3437-4309-aa32-3d2c974c315a None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:53.183744 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a9124e-3437-4309-aa32-3d2c974c315a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:53.197219 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a9124e-3437-4309-aa32-3d2c974c315a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:53.199380 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a9124e-3437-4309-aa32-3d2c974c315a None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:53.216241 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:53.222883 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-58a9124e-3437-4309-aa32-3d2c974c315a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:34:53.222883 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:53.222883 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:53.222883 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:53.222883 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:53.222883 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:53.222883 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:53.222883 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:53.222883 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:53.222883 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:53.222883 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:53.222883 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:53.222883 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:34:53.222883 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:34:53.222883 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:34:53.222883 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:34:53.223854 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:53.223854 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:53.223854 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:53.223854 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:53.223854 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:53.223854 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:53.223854 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:53.223854 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:53.223854 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:53.223854 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:53.223854 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:53.223854 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:53.223854 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:53.223854 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:53.223854 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:53.223854 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:53.223854 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:34:53.224691 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:34:53.224691 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:34:53.224691 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:53.226779 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 233/472] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:34:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:53.230805 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab30d9b-6dc0-46fe-8589-e90362de3fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:53.231311 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab30d9b-6dc0-46fe-8589-e90362de3fc6 None admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:53.231594 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab30d9b-6dc0-46fe-8589-e90362de3fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:53.231871 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab30d9b-6dc0-46fe-8589-e90362de3fc6 None admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:53.232536 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab30d9b-6dc0-46fe-8589-e90362de3fc6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:53.234247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a9124e-3437-4309-aa32-3d2c974c315a None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:53.237976 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab30d9b-6dc0-46fe-8589-e90362de3fc6 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:53.281692 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 240/473] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 10 22:34:53 2022] POST /identity/v3/users => generated 248 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:34:53.366638 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:53.371811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a9124e-3437-4309-aa32-3d2c974c315a None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:53.380369 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f0883-3b82-469e-8948-8e84b04ad484 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:53.380683 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f0883-3b82-469e-8948-8e84b04ad484 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:53.380991 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f0883-3b82-469e-8948-8e84b04ad484 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:53.382762 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 241/474] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:34:53.389815 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b79b6-9ef3-4bfe-933b-0b1a126282b8 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:53.390134 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b79b6-9ef3-4bfe-933b-0b1a126282b8 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:53.390376 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b79b6-9ef3-4bfe-933b-0b1a126282b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:53.414901 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450bcb3e-4342-4658-98ff-f089e5709685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:53.415129 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-4b6b79b6-9ef3-4bfe-933b-0b1a126282b8 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:53.415533 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bcb3e-4342-4658-98ff-f089e5709685 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:53.415824 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bcb3e-4342-4658-98ff-f089e5709685 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:53.416110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bcb3e-4342-4658-98ff-f089e5709685 None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:53.416208 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-4b6b79b6-9ef3-4bfe-933b-0b1a126282b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:53.416836 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450bcb3e-4342-4658-98ff-f089e5709685 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:53.417150 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450bcb3e-4342-4658-98ff-f089e5709685 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:53.419303 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450bcb3e-4342-4658-98ff-f089e5709685 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:53.435015 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 234/475] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:34:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:34:53.445791 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450bcb3e-4342-4658-98ff-f089e5709685 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:53.535083 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 242/476] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:34:53.562400 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6b79b6-9ef3-4bfe-933b-0b1a126282b8 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:53.603720 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450bcb3e-4342-4658-98ff-f089e5709685 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:53.652027 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8424e02-ee14-4c33-b86b-ec33287d417f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:53.652652 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8424e02-ee14-4c33-b86b-ec33287d417f None admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:53.652981 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8424e02-ee14-4c33-b86b-ec33287d417f None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:53.653261 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8424e02-ee14-4c33-b86b-ec33287d417f None admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:53.653934 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8424e02-ee14-4c33-b86b-ec33287d417f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:53.655914 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8424e02-ee14-4c33-b86b-ec33287d417f None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:53.685765 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 235/477] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 10 22:34:53 2022] POST /identity/v3/users => generated 249 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:34:53.686754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6b79b6-9ef3-4bfe-933b-0b1a126282b8 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:53.729999 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:53.762982 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f489d-8a5d-4557-b51d-08e8f98b0344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:53.763617 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f489d-8a5d-4557-b51d-08e8f98b0344 None admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:53.763894 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f489d-8a5d-4557-b51d-08e8f98b0344 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:53.764163 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f489d-8a5d-4557-b51d-08e8f98b0344 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:53.764832 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2f489d-8a5d-4557-b51d-08e8f98b0344 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:53.766282 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2f489d-8a5d-4557-b51d-08e8f98b0344 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:53.781954 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.sql.core [None req-5f2f489d-8a5d-4557-b51d-08e8f98b0344 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 22:34:53.781954 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 10 22:34:53.781954 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [parameters: {'id': '9bbc71418e804c95878d3b815a68c9ee', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 10 22:34:53.781954 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93439) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 22:34:53.808026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-5f2f489d-8a5d-4557-b51d-08e8f98b0344 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 10 22:34:53.808026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:53.808026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 22:34:53.808026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 22:34:53.808026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 22:34:53.808026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 22:34:53.808026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 22:34:53.808026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 22:34:53.808026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 22:34:53.808026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application conn.query(q) Aug 10 22:34:53.808026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 22:34:53.808026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 22:34:53.808026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 22:34:53.808026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application result.read() Aug 10 22:34:53.808026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 22:34:53.808929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 22:34:53.808929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 22:34:53.808929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 22:34:53.808929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 22:34:53.808929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 22:34:53.808929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 22:34:53.808929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 22:34:53.808929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 22:34:53.808929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:53.808929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 22:34:53.808929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:53.808929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:53.808929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 22:34:53.808929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 22:34:53.808929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 10 22:34:53.808929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return ref.to_dict() Aug 10 22:34:53.808929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 22:34:53.808929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application next(self.gen) Aug 10 22:34:53.810410 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 22:34:53.810410 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application yield resource Aug 10 22:34:53.810410 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 22:34:53.810410 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application next(self.gen) Aug 10 22:34:53.810410 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 22:34:53.810410 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 22:34:53.810410 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 22:34:53.810410 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 22:34:53.810410 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 22:34:53.810410 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise self.value Aug 10 22:34:53.810410 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 22:34:53.810410 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 22:34:53.810410 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 22:34:53.810410 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application session.commit() Aug 10 22:34:53.810410 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 22:34:53.810410 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 22:34:53.810410 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 22:34:53.811281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 22:34:53.811281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 22:34:53.811281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self.session.flush() Aug 10 22:34:53.811281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 22:34:53.811281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 22:34:53.811281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 22:34:53.811281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 22:34:53.811281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 22:34:53.811281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application compat.raise_( Aug 10 22:34:53.811281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 22:34:53.811281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception Aug 10 22:34:53.811281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 22:34:53.811281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 22:34:53.811281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 22:34:53.811281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 22:34:53.811281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 22:34:53.811281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 22:34:53.812159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 22:34:53.812159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 22:34:53.812159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 10 22:34:53.812159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 10 22:34:53.812159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 22:34:53.812159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 22:34:53.812159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 22:34:53.812159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 22:34:53.812159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 22:34:53.812159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 22:34:53.812159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 22:34:53.812159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 22:34:53.812159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 22:34:53.812159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 22:34:53.812159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 22:34:53.812159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception Aug 10 22:34:53.812952 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 22:34:53.812952 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 22:34:53.812952 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 22:34:53.812952 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 22:34:53.812952 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 22:34:53.812952 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 22:34:53.812952 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 22:34:53.812952 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application conn.query(q) Aug 10 22:34:53.812952 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 22:34:53.812952 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 22:34:53.812952 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 22:34:53.812952 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application result.read() Aug 10 22:34:53.812952 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 22:34:53.812952 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 22:34:53.812952 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 22:34:53.812952 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 22:34:53.814409 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 22:34:53.814409 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 22:34:53.814409 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 22:34:53.814409 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 22:34:53.814409 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 22:34:53.814409 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 10 22:34:53.814409 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [parameters: {'id': '9bbc71418e804c95878d3b815a68c9ee', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 10 22:34:53.814409 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 22:34:53.814409 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:53.814409 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:34:53.814409 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:53.814409 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:53.814409 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:53.814409 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:53.814409 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:53.814409 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:53.815193 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:53.815193 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:53.815193 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:53.815193 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:53.815193 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:53.815193 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:53.815193 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 10 22:34:53.815193 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 10 22:34:53.815193 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:53.815193 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:53.815193 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 10 22:34:53.815193 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 10 22:34:53.815193 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 22:34:53.815193 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 22:34:53.815193 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 10 22:34:53.815193 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:53.821714 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 243/478] 10.222.0.76 () {60 vars in 1131 bytes} [Wed Aug 10 22:34:53 2022] POST /identity/v3/roles => generated 144 bytes in 259 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:34:53.825748 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8424e02-ee14-4c33-b86b-ec33287d417f None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:53.879528 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:54.012978 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8424e02-ee14-4c33-b86b-ec33287d417f None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:54.049379 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cea010-00b3-46c5-b63e-3ffdee2837f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:54.050045 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cea010-00b3-46c5-b63e-3ffdee2837f0 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:54.050327 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cea010-00b3-46c5-b63e-3ffdee2837f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:54.050600 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cea010-00b3-46c5-b63e-3ffdee2837f0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:54.062936 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cea010-00b3-46c5-b63e-3ffdee2837f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:54.068205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-73cea010-00b3-46c5-b63e-3ffdee2837f0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:34:54.068205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:54.068205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:54.068205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:54.068205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:54.068205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:54.068205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:54.068205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:54.068205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:54.068205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:54.068205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:54.068205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:54.068205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:54.068205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:54.068205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:54.068205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:54.070311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:54.070311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:54.070311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:54.070311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:54.070311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:54.070311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:54.070311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:54.070311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:54.070311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:54.070311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:54.070311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:54.070311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:54.070311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:54.070311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:54.070311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:54.070311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:54.072268 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:54.072268 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:54.072268 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:54.072268 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:54.072268 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:54.072268 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:54.072268 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:54.072268 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:54.072268 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:54.072268 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:54.072268 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 22:34:54.072268 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:54.072268 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 236/479] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 10 22:34:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:54.078794 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f489d-8a5d-4557-b51d-08e8f98b0344 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:54.206849 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f489d-8a5d-4557-b51d-08e8f98b0344 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:54.244557 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa396318-258a-4782-9e21-56c686bb0c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:54.245183 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa396318-258a-4782-9e21-56c686bb0c03 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:54.245551 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa396318-258a-4782-9e21-56c686bb0c03 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:54.245819 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa396318-258a-4782-9e21-56c686bb0c03 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:54.246547 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa396318-258a-4782-9e21-56c686bb0c03 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:54.246840 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa396318-258a-4782-9e21-56c686bb0c03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:54.248210 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa396318-258a-4782-9e21-56c686bb0c03 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:54.257769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 244/480] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:34:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:55.206098 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb8a26-6cf5-49b1-a4c0-99ab7caf02b8 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:55.206732 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb8a26-6cf5-49b1-a4c0-99ab7caf02b8 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:55.206732 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb8a26-6cf5-49b1-a4c0-99ab7caf02b8 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:55.208400 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 237/481] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:55.213934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26286d8f-5251-44f3-804f-84bdcc5ba8ab None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:55.214346 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26286d8f-5251-44f3-804f-84bdcc5ba8ab None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:55.214542 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26286d8f-5251-44f3-804f-84bdcc5ba8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:55.238754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-26286d8f-5251-44f3-804f-84bdcc5ba8ab None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:55.239942 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-26286d8f-5251-44f3-804f-84bdcc5ba8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:55.276694 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:55.318555 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 245/482] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:55.360710 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb8a26-6cf5-49b1-a4c0-99ab7caf02b8 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:55.471784 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:55.507213 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb8a26-6cf5-49b1-a4c0-99ab7caf02b8 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:55.554707 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a28780-e745-4487-a926-73d863344e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:55.555339 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a28780-e745-4487-a926-73d863344e81 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:55.555614 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a28780-e745-4487-a926-73d863344e81 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:55.555881 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a28780-e745-4487-a926-73d863344e81 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:55.566491 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a28780-e745-4487-a926-73d863344e81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:55.571616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-44a28780-e745-4487-a926-73d863344e81 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:34:55.571616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:55.571616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:55.571616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:55.571616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:55.571616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:55.571616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:55.571616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:55.571616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:55.571616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:55.571616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:55.571616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:55.571616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:55.571616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:55.571616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:55.571616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:55.572861 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:55.572861 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:55.572861 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:55.572861 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:55.572861 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:55.572861 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:55.572861 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:55.572861 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:55.572861 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:55.572861 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:55.572861 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:55.572861 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:55.572861 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:55.572861 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:55.572861 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:55.572861 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:55.573694 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:55.573694 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:55.573694 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:55.573694 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:55.573694 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:55.573694 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:55.573694 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:55.573694 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:55.573694 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:55.573694 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:55.573694 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:34:55.573694 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:55.577386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 238/483] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 10 22:34:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:55.579130 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26286d8f-5251-44f3-804f-84bdcc5ba8ab None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:55.676587 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42e799-dadd-41d6-a79b-01eb0a00589e None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:55.676870 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42e799-dadd-41d6-a79b-01eb0a00589e None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:55.677147 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42e799-dadd-41d6-a79b-01eb0a00589e None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:55.678751 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 239/484] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:34:55.683697 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd0fd0-19db-41ba-9246-959336936a6a None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:55.683982 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd0fd0-19db-41ba-9246-959336936a6a None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:55.684260 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd0fd0-19db-41ba-9246-959336936a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:55.694772 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26286d8f-5251-44f3-804f-84bdcc5ba8ab None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:55.722587 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-42cd0fd0-19db-41ba-9246-959336936a6a None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:55.723731 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-42cd0fd0-19db-41ba-9246-959336936a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:55.741731 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a04f90-2049-4922-a4b4-8e2b6f600aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:55.742790 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a04f90-2049-4922-a4b4-8e2b6f600aa9 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:55.743263 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a04f90-2049-4922-a4b4-8e2b6f600aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:55.743728 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a04f90-2049-4922-a4b4-8e2b6f600aa9 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:55.744580 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a04f90-2049-4922-a4b4-8e2b6f600aa9 None admin] RBAC: Adding query filter params (name=service) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:55.745089 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a04f90-2049-4922-a4b4-8e2b6f600aa9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:55.746674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a04f90-2049-4922-a4b4-8e2b6f600aa9 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:55.761455 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 246/485] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 10 22:34:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:55.763852 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a04f90-2049-4922-a4b4-8e2b6f600aa9 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:55.832319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:55.876134 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 240/486] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:34:55.910072 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd0fd0-19db-41ba-9246-959336936a6a None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:55.948782 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a04f90-2049-4922-a4b4-8e2b6f600aa9 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:55.989232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:56.025989 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4095ac-fe27-4b45-8b51-ad0b8b90e5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:56.026643 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4095ac-fe27-4b45-8b51-ad0b8b90e5bc None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:56.026921 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4095ac-fe27-4b45-8b51-ad0b8b90e5bc None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:56.027191 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4095ac-fe27-4b45-8b51-ad0b8b90e5bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:56.029218 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd0fd0-19db-41ba-9246-959336936a6a None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:56.043091 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4095ac-fe27-4b45-8b51-ad0b8b90e5bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:56.044656 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4095ac-fe27-4b45-8b51-ad0b8b90e5bc None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:56.061310 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-2f4095ac-fe27-4b45-8b51-ad0b8b90e5bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:34:56.061310 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:56.061310 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:56.061310 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:56.061310 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:56.061310 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:56.061310 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:56.061310 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:56.061310 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:56.061310 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:56.061310 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:56.061310 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:56.061310 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:34:56.061310 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:34:56.061310 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:34:56.061310 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:34:56.062243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:56.062243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:56.062243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:56.062243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:56.062243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:56.062243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:56.062243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:56.062243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:56.062243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:56.062243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:56.062243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:56.062243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:56.062243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:56.062243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:56.062243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:56.062243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:56.062243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:34:56.066591 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:34:56.066591 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:34:56.066591 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:56.066591 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 247/487] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:34:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:34:56.066591 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591bbcd-ce1b-413b-8c69-6c8540375c49 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:56.066591 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591bbcd-ce1b-413b-8c69-6c8540375c49 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:56.066591 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591bbcd-ce1b-413b-8c69-6c8540375c49 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:56.066591 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 248/488] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:56.072121 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd42af-ec87-4849-8b02-ec78671a0941 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:56.072658 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd42af-ec87-4849-8b02-ec78671a0941 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:56.072955 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd42af-ec87-4849-8b02-ec78671a0941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:56.076198 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2c1a6f-0124-4998-b9c4-93390a0e7468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:56.076996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c1a6f-0124-4998-b9c4-93390a0e7468 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:56.077459 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c1a6f-0124-4998-b9c4-93390a0e7468 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:56.077877 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c1a6f-0124-4998-b9c4-93390a0e7468 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:56.090172 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2c1a6f-0124-4998-b9c4-93390a0e7468 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:56.094430 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-c8cd42af-ec87-4849-8b02-ec78671a0941 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:56.096331 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-c8cd42af-ec87-4849-8b02-ec78671a0941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:56.097788 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-8b2c1a6f-0124-4998-b9c4-93390a0e7468 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:34:56.097788 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:56.097788 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:56.097788 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:56.097788 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:56.097788 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:56.097788 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:56.097788 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:56.097788 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:56.097788 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:56.097788 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:56.097788 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:56.097788 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:56.097788 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:56.097788 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:56.097788 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:56.098879 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:56.098879 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:56.098879 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:56.098879 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:56.098879 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:56.098879 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:56.098879 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:56.098879 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:56.098879 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:56.098879 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:56.098879 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:56.098879 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:56.098879 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:56.098879 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:56.098879 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:56.098879 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:56.099897 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:56.099897 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:56.099897 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:56.099897 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:56.099897 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:56.099897 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:56.099897 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:56.099897 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:56.099897 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:56.099897 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:56.099897 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:34:56.099897 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:56.102075 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 241/489] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 10 22:34:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:34:56.103108 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2c1a6f-0124-4998-b9c4-93390a0e7468 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:56.190679 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 249/490] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:56.192170 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cd42af-ec87-4849-8b02-ec78671a0941 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:56.213091 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:56.243416 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2c1a6f-0124-4998-b9c4-93390a0e7468 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:56.271465 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:56.281307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e964ef-18ec-416e-b71c-19571bd388ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:56.282039 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e964ef-18ec-416e-b71c-19571bd388ba None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:56.282397 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e964ef-18ec-416e-b71c-19571bd388ba None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:56.282746 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e964ef-18ec-416e-b71c-19571bd388ba None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:56.283676 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e964ef-18ec-416e-b71c-19571bd388ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:56.283982 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e964ef-18ec-416e-b71c-19571bd388ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:56.285410 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e964ef-18ec-416e-b71c-19571bd388ba None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:56.298000 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 242/491] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:34:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:56.302171 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e964ef-18ec-416e-b71c-19571bd388ba None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:56.322136 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cd42af-ec87-4849-8b02-ec78671a0941 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:56.370309 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27598294-0ef8-4072-adcf-e7ad70d37165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:56.371753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27598294-0ef8-4072-adcf-e7ad70d37165 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:56.372180 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27598294-0ef8-4072-adcf-e7ad70d37165 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:56.372891 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27598294-0ef8-4072-adcf-e7ad70d37165 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:56.373728 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27598294-0ef8-4072-adcf-e7ad70d37165 None admin] RBAC: Adding query filter params (name=service) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:56.374470 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27598294-0ef8-4072-adcf-e7ad70d37165 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:56.376774 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27598294-0ef8-4072-adcf-e7ad70d37165 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:56.393230 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 250/492] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 10 22:34:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:56.393230 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27598294-0ef8-4072-adcf-e7ad70d37165 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:56.432140 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e964ef-18ec-416e-b71c-19571bd388ba None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:56.473545 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b6673d-8d83-4589-8562-cde944c111a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:56.474633 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6673d-8d83-4589-8562-cde944c111a0 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:56.475235 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6673d-8d83-4589-8562-cde944c111a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:56.475710 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6673d-8d83-4589-8562-cde944c111a0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:56.488252 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b6673d-8d83-4589-8562-cde944c111a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:56.493675 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-25b6673d-8d83-4589-8562-cde944c111a0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:34:56.493675 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:56.493675 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:56.493675 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:56.493675 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:56.493675 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:56.493675 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:56.493675 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:56.493675 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:56.493675 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:56.493675 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:56.493675 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:56.493675 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:56.493675 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:56.493675 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:56.493675 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:56.494532 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:56.494532 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:56.494532 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:56.494532 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:56.494532 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:56.494532 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:56.494532 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:56.494532 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:56.494532 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:56.494532 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:56.494532 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:56.494532 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:56.494532 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:56.494532 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:56.494532 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:56.494532 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:56.495335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:56.495335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:56.495335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:56.495335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:56.495335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:56.495335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:56.495335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:56.495335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:56.495335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:56.495335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:56.495335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:34:56.495335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:56.497333 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:56.503282 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 243/493] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 10 22:34:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:56.504480 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b6673d-8d83-4589-8562-cde944c111a0 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:56.522255 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27598294-0ef8-4072-adcf-e7ad70d37165 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:56.561092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:56.574058 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af01bbd-9ae4-4b82-a9d3-0fc6da27fdcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:56.575079 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af01bbd-9ae4-4b82-a9d3-0fc6da27fdcf None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:56.575374 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af01bbd-9ae4-4b82-a9d3-0fc6da27fdcf None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:56.575651 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af01bbd-9ae4-4b82-a9d3-0fc6da27fdcf None admin] PATH_INFO: `/v3/users/nova` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:56.585529 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af01bbd-9ae4-4b82-a9d3-0fc6da27fdcf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:56.587497 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af01bbd-9ae4-4b82-a9d3-0fc6da27fdcf None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:56.603723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-4af01bbd-9ae4-4b82-a9d3-0fc6da27fdcf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:34:56.603723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:56.603723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:56.603723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:56.603723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:56.603723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:56.603723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:56.603723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:56.603723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:56.603723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:56.603723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:56.603723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:56.603723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:34:56.603723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:34:56.603723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:34:56.603723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:34:56.604691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:56.604691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:56.604691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:34:56.604691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:34:56.604691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:34:56.604691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:34:56.604691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:56.604691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:56.604691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:56.604691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:56.604691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:56.604691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:56.604691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:56.604691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:56.604691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:56.604691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:56.604691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:56.604691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:56.605620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:56.605620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:56.605620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:34:56.605620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:34:56.605620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:34:56.605620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:34:56.605620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:34:56.605620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:34:56.605620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:34:56.605620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:56.605620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 251/494] 10.222.0.76 () {56 vars in 1093 bytes} [Wed Aug 10 22:34:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:56.606188 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af01bbd-9ae4-4b82-a9d3-0fc6da27fdcf None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:56.631909 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b6673d-8d83-4589-8562-cde944c111a0 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:56.671197 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5148a-3fb4-441c-973a-1b99cfa02bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:56.672098 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5148a-3fb4-441c-973a-1b99cfa02bb1 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:56.672313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5148a-3fb4-441c-973a-1b99cfa02bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:56.672573 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5148a-3fb4-441c-973a-1b99cfa02bb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:56.684154 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae5148a-3fb4-441c-973a-1b99cfa02bb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:56.685639 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae5148a-3fb4-441c-973a-1b99cfa02bb1 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:56.702355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-cae5148a-3fb4-441c-973a-1b99cfa02bb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:34:56.702355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:56.702355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:56.702355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:56.702355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:56.702355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:56.702355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:56.702355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:56.702355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:56.702355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:56.702355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:56.702355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:56.702355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:34:56.702355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:34:56.702355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:34:56.702355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:34:56.703301 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:56.703301 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:56.703301 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:56.703301 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:56.703301 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:56.703301 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:56.703301 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:56.703301 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:56.703301 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:56.703301 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:56.703301 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:56.703301 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:56.703301 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:56.703301 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:56.703301 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:56.703301 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:56.703301 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:34:56.704168 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:34:56.704168 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:34:56.704168 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:56.704168 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 244/495] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:34:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:56.704391 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5148a-3fb4-441c-973a-1b99cfa02bb1 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:56.723938 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af01bbd-9ae4-4b82-a9d3-0fc6da27fdcf None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:56.759946 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a187a1b-c0d0-49d4-929a-d4a5e26778b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:56.760472 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a187a1b-c0d0-49d4-929a-d4a5e26778b9 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:56.760754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a187a1b-c0d0-49d4-929a-d4a5e26778b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:56.761025 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a187a1b-c0d0-49d4-929a-d4a5e26778b9 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:56.761726 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a187a1b-c0d0-49d4-929a-d4a5e26778b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:56.762027 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a187a1b-c0d0-49d4-929a-d4a5e26778b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:56.763411 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a187a1b-c0d0-49d4-929a-d4a5e26778b9 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:56.769401 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:56.773456 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 252/496] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 10 22:34:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:56.774719 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ac4de-29da-465a-8ca6-ac68b13fbc22 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:56.775006 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ac4de-29da-465a-8ca6-ac68b13fbc22 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:56.775279 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ac4de-29da-465a-8ca6-ac68b13fbc22 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:56.776651 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 253/497] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:56.777154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ac4de-29da-465a-8ca6-ac68b13fbc22 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:56.819775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:56.825003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5148a-3fb4-441c-973a-1b99cfa02bb1 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:56.862017 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c95fb-c3f9-407e-b16f-3854a76a1981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:56.862641 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c95fb-c3f9-407e-b16f-3854a76a1981 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:56.862923 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c95fb-c3f9-407e-b16f-3854a76a1981 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:56.863196 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c95fb-c3f9-407e-b16f-3854a76a1981 None admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:56.863922 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526c95fb-c3f9-407e-b16f-3854a76a1981 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:56.864228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526c95fb-c3f9-407e-b16f-3854a76a1981 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:56.865685 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526c95fb-c3f9-407e-b16f-3854a76a1981 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:56.889931 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ac4de-29da-465a-8ca6-ac68b13fbc22 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:56.903787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 245/498] 10.222.0.76 () {56 vars in 1133 bytes} [Wed Aug 10 22:34:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:56.904638 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c95fb-c3f9-407e-b16f-3854a76a1981 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:56.929849 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb3d8a-7676-4f06-b48f-a76ff0be4f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:56.930468 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb3d8a-7676-4f06-b48f-a76ff0be4f4f None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:56.930746 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb3d8a-7676-4f06-b48f-a76ff0be4f4f None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:56.931015 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb3d8a-7676-4f06-b48f-a76ff0be4f4f None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:56.931651 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabb3d8a-7676-4f06-b48f-a76ff0be4f4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:56.931950 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabb3d8a-7676-4f06-b48f-a76ff0be4f4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:56.933377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabb3d8a-7676-4f06-b48f-a76ff0be4f4f None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:56.945632 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 254/499] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:34:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:56.946839 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4e966-c125-4128-b6bd-54a327b4f508 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:56.947126 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4e966-c125-4128-b6bd-54a327b4f508 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:56.947390 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4e966-c125-4128-b6bd-54a327b4f508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:56.965616 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-2ef4e966-c125-4128-b6bd-54a327b4f508 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:56.966802 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-2ef4e966-c125-4128-b6bd-54a327b4f508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:57.006812 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c95fb-c3f9-407e-b16f-3854a76a1981 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:57.023374 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:57.035244 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 255/500] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:57.036636 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36635661-7691-4216-a544-f5eea2e5d16c None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:57.036918 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36635661-7691-4216-a544-f5eea2e5d16c None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:57.037195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36635661-7691-4216-a544-f5eea2e5d16c None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:57.038721 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 256/501] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:34:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:34:57.041454 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314fc33e-cbec-43a7-a1cf-c3f4ecacc45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:57.042078 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fc33e-cbec-43a7-a1cf-c3f4ecacc45f None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:57.042366 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fc33e-cbec-43a7-a1cf-c3f4ecacc45f None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:57.042930 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fc33e-cbec-43a7-a1cf-c3f4ecacc45f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:57.043280 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36635661-7691-4216-a544-f5eea2e5d16c None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:57.053654 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314fc33e-cbec-43a7-a1cf-c3f4ecacc45f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:57.055124 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314fc33e-cbec-43a7-a1cf-c3f4ecacc45f None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:57.070327 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-314fc33e-cbec-43a7-a1cf-c3f4ecacc45f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:34:57.070327 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:57.070327 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:57.070327 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:57.070327 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:57.070327 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:57.070327 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:57.070327 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:57.070327 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:57.070327 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:57.070327 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:57.070327 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:57.070327 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:34:57.070327 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:34:57.070327 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:34:57.070327 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:34:57.071313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:57.071313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:57.071313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:57.071313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:57.071313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:57.071313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:57.071313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:57.071313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:57.071313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:57.071313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:57.071313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:57.071313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:57.071313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:57.071313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:57.071313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:57.071313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:57.071313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:34:57.072160 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:34:57.072160 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:34:57.072160 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:57.072160 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 246/502] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:34:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:57.072370 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314fc33e-cbec-43a7-a1cf-c3f4ecacc45f None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:57.084351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:57.159709 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36635661-7691-4216-a544-f5eea2e5d16c None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:57.181894 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314fc33e-cbec-43a7-a1cf-c3f4ecacc45f None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:57.193151 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ee796-8444-4ebb-a72c-504e9882cb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:57.193778 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ee796-8444-4ebb-a72c-504e9882cb0d None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:57.194058 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ee796-8444-4ebb-a72c-504e9882cb0d None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:57.194333 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ee796-8444-4ebb-a72c-504e9882cb0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:57.218012 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84893c90-b8e0-4ffd-bfdd-7bf766073da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:57.218650 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84893c90-b8e0-4ffd-bfdd-7bf766073da8 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:57.218931 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84893c90-b8e0-4ffd-bfdd-7bf766073da8 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:57.219204 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84893c90-b8e0-4ffd-bfdd-7bf766073da8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:57.227722 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84893c90-b8e0-4ffd-bfdd-7bf766073da8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:57.229343 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84893c90-b8e0-4ffd-bfdd-7bf766073da8 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:57.242058 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-84893c90-b8e0-4ffd-bfdd-7bf766073da8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 22:34:57.242058 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:57.242058 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:57.242058 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:57.242058 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:57.242058 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:57.242058 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:57.242058 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:57.242058 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:57.242058 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:57.242058 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:57.242058 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:57.242058 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:34:57.242058 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:34:57.242058 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:34:57.242058 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:34:57.243148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:57.243148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:57.243148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:34:57.243148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:34:57.243148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:34:57.243148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:34:57.243148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:57.243148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:57.243148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:57.243148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:57.243148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:57.243148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:57.243148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:57.243148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:57.243148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:57.243148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:57.243148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:57.243148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:57.244586 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:57.244586 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:57.244586 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:34:57.244586 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:34:57.244586 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:34:57.244586 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:34:57.244586 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:34:57.244586 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:34:57.244586 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 22:34:57.244586 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:57.244586 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 247/503] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:57.244586 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a64671-a41f-4761-be9a-adb29bf1896d None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:57.245196 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a64671-a41f-4761-be9a-adb29bf1896d None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:57.245196 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a64671-a41f-4761-be9a-adb29bf1896d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:57.260032 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:57.263419 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-66a64671-a41f-4761-be9a-adb29bf1896d None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:34:57.264634 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-66a64671-a41f-4761-be9a-adb29bf1896d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:34:57.271716 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ee796-8444-4ebb-a72c-504e9882cb0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:57.273184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ee796-8444-4ebb-a72c-504e9882cb0d None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:57.327168 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:57.333253 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 248/504] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:34:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:34:57.333909 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a64671-a41f-4761-be9a-adb29bf1896d None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:57.387480 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 257/505] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:34:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:34:57.388891 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ee796-8444-4ebb-a72c-504e9882cb0d None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:57.439485 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a64671-a41f-4761-be9a-adb29bf1896d None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:57.475563 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a49975-6af3-48f5-b00f-2cbcecfc5a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:57.476221 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a49975-6af3-48f5-b00f-2cbcecfc5a6b None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:57.476504 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a49975-6af3-48f5-b00f-2cbcecfc5a6b None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:57.476778 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a49975-6af3-48f5-b00f-2cbcecfc5a6b None admin] PATH_INFO: `/v3/roles/service` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:57.480009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:57.487599 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a49975-6af3-48f5-b00f-2cbcecfc5a6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:34:57.491352 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ee796-8444-4ebb-a72c-504e9882cb0d None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:57.492335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-d9a49975-6af3-48f5-b00f-2cbcecfc5a6b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:34:57.492335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:57.492335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:57.492335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:57.492335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:57.492335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:57.492335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:57.492335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:57.492335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:57.492335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:57.492335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:57.492335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:57.492335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:34:57.492335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:34:57.492335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:34:57.492335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:34:57.493519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:34:57.493519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:34:57.493519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:34:57.493519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:34:57.493519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:57.493519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:57.493519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:57.493519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:57.493519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:57.493519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:57.493519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:57.493519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:57.493519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:57.493519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:57.493519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:57.493519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:57.494407 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:57.494407 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:57.494407 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:57.494407 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:57.494407 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:34:57.494407 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:34:57.494407 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:34:57.494407 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:34:57.494407 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:34:57.494407 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:34:57.494407 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:34:57.494407 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:57.494407 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 249/506] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 10 22:34:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:57.494407 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a49975-6af3-48f5-b00f-2cbcecfc5a6b None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:57.531887 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f828ff-5006-4260-8b60-bd546ff51f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:57.532498 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f828ff-5006-4260-8b60-bd546ff51f30 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:57.532782 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f828ff-5006-4260-8b60-bd546ff51f30 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:57.533050 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f828ff-5006-4260-8b60-bd546ff51f30 None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:57.533936 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f828ff-5006-4260-8b60-bd546ff51f30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:57.534264 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f828ff-5006-4260-8b60-bd546ff51f30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:57.535924 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f828ff-5006-4260-8b60-bd546ff51f30 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:57.547380 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 258/507] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:34:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:57.548188 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f828ff-5006-4260-8b60-bd546ff51f30 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:57.569540 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:57.614525 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a49975-6af3-48f5-b00f-2cbcecfc5a6b None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:57.647356 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf1ff0-20fa-4019-bcfc-92b132096438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:57.647968 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf1ff0-20fa-4019-bcfc-92b132096438 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:57.648226 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf1ff0-20fa-4019-bcfc-92b132096438 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:57.648502 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf1ff0-20fa-4019-bcfc-92b132096438 None admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:57.649224 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cf1ff0-20fa-4019-bcfc-92b132096438 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:57.649562 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cf1ff0-20fa-4019-bcfc-92b132096438 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:57.651004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cf1ff0-20fa-4019-bcfc-92b132096438 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:57.652224 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f828ff-5006-4260-8b60-bd546ff51f30 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:57.668860 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 250/508] 10.222.0.76 () {56 vars in 1137 bytes} [Wed Aug 10 22:34:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:57.669525 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf1ff0-20fa-4019-bcfc-92b132096438 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:57.687308 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cffc74-99ed-43ee-90c8-2bc9646f51fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:57.688074 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cffc74-99ed-43ee-90c8-2bc9646f51fa None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:57.688354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cffc74-99ed-43ee-90c8-2bc9646f51fa None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:57.688621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cffc74-99ed-43ee-90c8-2bc9646f51fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:57.699502 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cffc74-99ed-43ee-90c8-2bc9646f51fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:57.701067 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cffc74-99ed-43ee-90c8-2bc9646f51fa None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:57.715025 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-37cffc74-99ed-43ee-90c8-2bc9646f51fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:34:57.715025 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:57.715025 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:57.715025 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:57.715025 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:57.715025 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:57.715025 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:57.715025 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:57.715025 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:57.715025 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:57.715025 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:57.715025 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:57.715025 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:34:57.715025 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:34:57.715025 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:34:57.715025 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:34:57.716082 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:57.716082 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:57.716082 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:57.716082 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:57.716082 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:57.716082 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:57.716082 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:57.716082 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:57.716082 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:57.716082 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:57.716082 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:57.716082 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:57.716082 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:57.716082 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:57.716082 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:57.716082 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:57.716082 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:34:57.717042 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:34:57.717042 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:34:57.717042 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:57.717042 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 259/509] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:34:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:57.722163 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cffc74-99ed-43ee-90c8-2bc9646f51fa None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:57.724995 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:57.770932 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf1ff0-20fa-4019-bcfc-92b132096438 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:57.787542 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:57.806294 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5204c-f8c7-4051-be58-31b4a1a574fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:57.806928 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5204c-f8c7-4051-be58-31b4a1a574fd None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:57.807211 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5204c-f8c7-4051-be58-31b4a1a574fd None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:57.807487 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5204c-f8c7-4051-be58-31b4a1a574fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:57.818066 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d5204c-f8c7-4051-be58-31b4a1a574fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:57.819512 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d5204c-f8c7-4051-be58-31b4a1a574fd None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:57.825013 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cffc74-99ed-43ee-90c8-2bc9646f51fa None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:57.834354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-58d5204c-f8c7-4051-be58-31b4a1a574fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:34:57.834354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:57.834354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:57.834354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:57.834354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:57.834354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:57.834354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:57.834354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:57.834354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:57.834354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:57.834354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:57.834354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:57.834354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:34:57.834354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:34:57.834354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:34:57.834354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:34:57.835450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:57.835450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:57.835450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:57.835450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:57.835450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:57.835450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:57.835450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:57.835450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:57.835450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:57.835450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:57.835450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:57.835450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:57.835450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:57.835450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:57.835450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:57.835450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:57.835450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:34:57.836481 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:34:57.836481 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:34:57.836481 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:57.836481 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 251/510] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:34:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:57.836481 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5204c-f8c7-4051-be58-31b4a1a574fd None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:57.862468 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f428ef20-21af-40c0-af88-e4729e88359d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:57.862905 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428ef20-21af-40c0-af88-e4729e88359d None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:57.863183 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428ef20-21af-40c0-af88-e4729e88359d None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:57.863450 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428ef20-21af-40c0-af88-e4729e88359d None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:57.864122 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f428ef20-21af-40c0-af88-e4729e88359d None admin] RBAC: Adding query filter params (name=service) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:57.864421 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f428ef20-21af-40c0-af88-e4729e88359d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:57.865905 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f428ef20-21af-40c0-af88-e4729e88359d None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:57.874276 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 260/511] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 10 22:34:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:57.875067 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f428ef20-21af-40c0-af88-e4729e88359d None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:57.936964 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:57.940288 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5204c-f8c7-4051-be58-31b4a1a574fd None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:57.975587 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a82634-ff49-4409-ac3c-5488bd54f413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:57.976326 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a82634-ff49-4409-ac3c-5488bd54f413 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:57.976609 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a82634-ff49-4409-ac3c-5488bd54f413 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:57.976885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a82634-ff49-4409-ac3c-5488bd54f413 None admin] PATH_INFO: `/v3/users/glance` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:57.977957 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f428ef20-21af-40c0-af88-e4729e88359d None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:57.983965 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a82634-ff49-4409-ac3c-5488bd54f413 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:57.985468 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a82634-ff49-4409-ac3c-5488bd54f413 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:57.996800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-47a82634-ff49-4409-ac3c-5488bd54f413 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 22:34:57.996800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:57.996800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:57.996800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:57.996800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:57.996800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:57.996800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:57.996800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:57.996800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:57.996800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:57.996800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:57.996800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:57.996800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:34:57.996800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:34:57.996800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:34:57.996800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:34:57.997924 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:57.997924 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:57.997924 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:34:57.997924 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:34:57.997924 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:34:57.997924 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:34:57.997924 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:57.997924 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:57.997924 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:57.997924 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:57.997924 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:57.997924 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:57.997924 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:57.997924 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:57.997924 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:57.997924 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:57.997924 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:57.997924 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:57.998991 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:57.998991 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:57.998991 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:34:57.998991 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:34:57.998991 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:34:57.998991 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:34:57.998991 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:34:57.998991 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:34:57.998991 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 22:34:57.998991 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:57.998991 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 252/512] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:34:57 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:57.998991 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a82634-ff49-4409-ac3c-5488bd54f413 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:58.013777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f38ab-2165-49c9-a50d-7c2bc7b9de4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:58.014381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f38ab-2165-49c9-a50d-7c2bc7b9de4e None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:58.014655 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f38ab-2165-49c9-a50d-7c2bc7b9de4e None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:58.014925 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f38ab-2165-49c9-a50d-7c2bc7b9de4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:58.032869 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:58.102115 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f38ab-2165-49c9-a50d-7c2bc7b9de4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:58.103561 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f38ab-2165-49c9-a50d-7c2bc7b9de4e None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:58.119370 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a82634-ff49-4409-ac3c-5488bd54f413 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:58.152417 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c2a508-4c23-4273-931b-5b69b9d8e591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:58.153017 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c2a508-4c23-4273-931b-5b69b9d8e591 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:58.153325 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c2a508-4c23-4273-931b-5b69b9d8e591 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:58.153602 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c2a508-4c23-4273-931b-5b69b9d8e591 None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:58.154233 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c2a508-4c23-4273-931b-5b69b9d8e591 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:58.154527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c2a508-4c23-4273-931b-5b69b9d8e591 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:58.155935 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c2a508-4c23-4273-931b-5b69b9d8e591 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:58.163930 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:58.167935 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 253/513] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:34:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:58.168731 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c2a508-4c23-4273-931b-5b69b9d8e591 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:58.207427 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 261/514] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:34:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:34:58.208274 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f38ab-2165-49c9-a50d-7c2bc7b9de4e None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:58.255568 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:58.267953 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c2a508-4c23-4273-931b-5b69b9d8e591 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:58.300824 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d84096-1054-46e9-bce9-78ee45b6bb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:58.301486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d84096-1054-46e9-bce9-78ee45b6bb9a None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:58.301758 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d84096-1054-46e9-bce9-78ee45b6bb9a None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:58.302026 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d84096-1054-46e9-bce9-78ee45b6bb9a None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:58.302656 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d84096-1054-46e9-bce9-78ee45b6bb9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:58.302954 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d84096-1054-46e9-bce9-78ee45b6bb9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:58.304322 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d84096-1054-46e9-bce9-78ee45b6bb9a None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:58.307197 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f38ab-2165-49c9-a50d-7c2bc7b9de4e None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:58.315604 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 254/515] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:34:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:58.316506 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d84096-1054-46e9-bce9-78ee45b6bb9a None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:58.343540 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd4416-9a2c-496b-8a0c-19b586dc45e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:58.344169 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd4416-9a2c-496b-8a0c-19b586dc45e5 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:58.344449 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd4416-9a2c-496b-8a0c-19b586dc45e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:58.344716 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd4416-9a2c-496b-8a0c-19b586dc45e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:58.355493 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fd4416-9a2c-496b-8a0c-19b586dc45e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:58.356939 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fd4416-9a2c-496b-8a0c-19b586dc45e5 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:58.372621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-96fd4416-9a2c-496b-8a0c-19b586dc45e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:34:58.372621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:58.372621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:58.372621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:58.372621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:58.372621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:58.372621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:58.372621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:58.372621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:58.372621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:58.372621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:58.372621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:58.372621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:34:58.372621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:34:58.372621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:34:58.372621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:34:58.373574 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:58.373574 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:58.373574 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:58.373574 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:58.373574 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:58.373574 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:58.373574 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:58.373574 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:58.373574 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:58.373574 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:58.373574 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:58.373574 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:58.373574 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:58.373574 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:58.373574 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:58.373574 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:58.373574 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:34:58.374411 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:34:58.374411 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:34:58.374411 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:58.374411 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 262/516] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:34:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:58.374411 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd4416-9a2c-496b-8a0c-19b586dc45e5 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:58.392225 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:58.425452 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d84096-1054-46e9-bce9-78ee45b6bb9a None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:58.459013 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746160c-1149-490f-8c0d-f67d44b52b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:58.459629 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746160c-1149-490f-8c0d-f67d44b52b49 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:58.459914 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746160c-1149-490f-8c0d-f67d44b52b49 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:58.460194 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746160c-1149-490f-8c0d-f67d44b52b49 None admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:58.460922 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746160c-1149-490f-8c0d-f67d44b52b49 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:58.461232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746160c-1149-490f-8c0d-f67d44b52b49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:58.462719 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746160c-1149-490f-8c0d-f67d44b52b49 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:58.476970 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd4416-9a2c-496b-8a0c-19b586dc45e5 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:58.481140 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 255/517] 10.222.0.76 () {56 vars in 1137 bytes} [Wed Aug 10 22:34:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:58.481989 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746160c-1149-490f-8c0d-f67d44b52b49 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:58.492683 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:58.511182 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1d751-7a11-4ce9-92ca-13df969ff13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:58.511801 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1d751-7a11-4ce9-92ca-13df969ff13e None admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:58.512080 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1d751-7a11-4ce9-92ca-13df969ff13e None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:58.512353 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1d751-7a11-4ce9-92ca-13df969ff13e None admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:58.512986 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1d751-7a11-4ce9-92ca-13df969ff13e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:58.514488 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1d751-7a11-4ce9-92ca-13df969ff13e None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:58.535514 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 263/518] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 10 22:34:58 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:34:58.536320 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1d751-7a11-4ce9-92ca-13df969ff13e None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:58.613050 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746160c-1149-490f-8c0d-f67d44b52b49 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:58.628701 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:58.653526 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dddf40c-b672-43b9-99a8-93ba7f82a72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:58.654177 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dddf40c-b672-43b9-99a8-93ba7f82a72f None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:58.654459 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dddf40c-b672-43b9-99a8-93ba7f82a72f None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:58.654771 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dddf40c-b672-43b9-99a8-93ba7f82a72f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:58.665834 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1d751-7a11-4ce9-92ca-13df969ff13e None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:58.667435 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dddf40c-b672-43b9-99a8-93ba7f82a72f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:58.669271 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dddf40c-b672-43b9-99a8-93ba7f82a72f None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:58.686250 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-2dddf40c-b672-43b9-99a8-93ba7f82a72f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:34:58.686250 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:58.686250 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:58.686250 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:58.686250 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:58.686250 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:58.686250 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:58.686250 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:58.686250 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:58.686250 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:58.686250 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:58.686250 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:58.686250 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:34:58.686250 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:34:58.686250 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:34:58.686250 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:34:58.687202 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:58.687202 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:58.687202 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:58.687202 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:58.687202 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:58.687202 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:58.687202 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:58.687202 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:58.687202 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:58.687202 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:58.687202 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:58.687202 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:58.687202 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:58.687202 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:58.687202 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:58.687202 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:58.687202 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:34:58.688043 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:34:58.688043 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:34:58.688043 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:58.688043 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 256/519] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:34:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:58.688705 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dddf40c-b672-43b9-99a8-93ba7f82a72f None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:58.709222 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18faa34-1436-422e-ae91-e48dd8d57160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:58.709906 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18faa34-1436-422e-ae91-e48dd8d57160 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:58.710186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18faa34-1436-422e-ae91-e48dd8d57160 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:58.710461 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18faa34-1436-422e-ae91-e48dd8d57160 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:58.719007 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18faa34-1436-422e-ae91-e48dd8d57160 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:58.720941 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18faa34-1436-422e-ae91-e48dd8d57160 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:58.738052 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-a18faa34-1436-422e-ae91-e48dd8d57160 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:34:58.738052 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:58.738052 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:58.738052 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:58.738052 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:58.738052 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:58.738052 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:58.738052 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:58.738052 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:58.738052 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:58.738052 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:58.738052 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:58.738052 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:34:58.738052 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:34:58.738052 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:34:58.739172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:34:58.739172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:58.739172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:58.739172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:58.739172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:58.739172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:58.739172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:58.739172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:58.739172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:58.739172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:58.739172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:58.739172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:58.739172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:58.739172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:58.739172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:58.739172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:58.739172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:58.740044 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:34:58.740044 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:34:58.740044 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:34:58.740044 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:34:58.740044 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:34:58.740044 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:34:58.740044 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:34:58.740044 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:58.740044 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 264/520] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:34:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:58.740513 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18faa34-1436-422e-ae91-e48dd8d57160 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:58.762547 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:58.810856 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dddf40c-b672-43b9-99a8-93ba7f82a72f None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:58.848249 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517348ec-77a8-496c-9a33-1ddc9cf3728e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:58.848868 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517348ec-77a8-496c-9a33-1ddc9cf3728e None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:58.849158 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517348ec-77a8-496c-9a33-1ddc9cf3728e None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:58.849472 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517348ec-77a8-496c-9a33-1ddc9cf3728e None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:58.850119 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517348ec-77a8-496c-9a33-1ddc9cf3728e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:58.850422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517348ec-77a8-496c-9a33-1ddc9cf3728e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:58.851827 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517348ec-77a8-496c-9a33-1ddc9cf3728e None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:58.861796 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18faa34-1436-422e-ae91-e48dd8d57160 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:58.864427 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 257/521] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:34:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:58.865242 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517348ec-77a8-496c-9a33-1ddc9cf3728e None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:58.900730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:58.907051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ced8a-7244-44fe-905f-9292c97975e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:58.907668 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ced8a-7244-44fe-905f-9292c97975e1 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:58.907949 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ced8a-7244-44fe-905f-9292c97975e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:58.908223 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ced8a-7244-44fe-905f-9292c97975e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:58.976282 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517348ec-77a8-496c-9a33-1ddc9cf3728e None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:58.983477 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ced8a-7244-44fe-905f-9292c97975e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:58.984939 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ced8a-7244-44fe-905f-9292c97975e1 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:58.996038 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:59.015763 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86d826-cb2e-40dd-81a2-fbb0a8cdcf1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:59.016374 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86d826-cb2e-40dd-81a2-fbb0a8cdcf1f None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:59.016656 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86d826-cb2e-40dd-81a2-fbb0a8cdcf1f None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:59.016926 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86d826-cb2e-40dd-81a2-fbb0a8cdcf1f None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:59.017611 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b86d826-cb2e-40dd-81a2-fbb0a8cdcf1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:59.017913 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b86d826-cb2e-40dd-81a2-fbb0a8cdcf1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:59.020098 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b86d826-cb2e-40dd-81a2-fbb0a8cdcf1f None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:59.033147 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 258/522] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:34:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:59.034691 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86d826-cb2e-40dd-81a2-fbb0a8cdcf1f None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:59.096047 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 265/523] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:34:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:34:59.097930 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ced8a-7244-44fe-905f-9292c97975e1 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:59.131674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:59.152280 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86d826-cb2e-40dd-81a2-fbb0a8cdcf1f None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:59.188153 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f49ece-4d5b-47ef-b2a9-6b966ec014f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:59.188760 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f49ece-4d5b-47ef-b2a9-6b966ec014f5 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:59.189038 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f49ece-4d5b-47ef-b2a9-6b966ec014f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:59.189334 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f49ece-4d5b-47ef-b2a9-6b966ec014f5 None admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:59.190036 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f49ece-4d5b-47ef-b2a9-6b966ec014f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:59.191282 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f49ece-4d5b-47ef-b2a9-6b966ec014f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:59.192720 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f49ece-4d5b-47ef-b2a9-6b966ec014f5 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:59.217267 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 259/524] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:34:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:59.217471 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ced8a-7244-44fe-905f-9292c97975e1 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:59.217566 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f49ece-4d5b-47ef-b2a9-6b966ec014f5 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:59.254907 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e352551e-b552-4778-852f-e885d9075b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:59.255539 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352551e-b552-4778-852f-e885d9075b54 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:59.255874 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352551e-b552-4778-852f-e885d9075b54 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:59.256152 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352551e-b552-4778-852f-e885d9075b54 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:59.259198 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:59.263808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e352551e-b552-4778-852f-e885d9075b54 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:59.265271 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e352551e-b552-4778-852f-e885d9075b54 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:59.278698 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-e352551e-b552-4778-852f-e885d9075b54 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 22:34:59.278698 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:59.278698 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:59.278698 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:59.278698 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:59.278698 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:59.278698 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:59.278698 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:59.278698 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:59.278698 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:59.278698 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:59.278698 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:59.278698 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:34:59.278698 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:34:59.278698 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:34:59.278698 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:34:59.279769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:59.279769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:59.279769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:34:59.279769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:34:59.279769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:34:59.279769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:34:59.279769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:59.279769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:59.279769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:59.279769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:59.279769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:59.279769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:59.279769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:59.279769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:59.279769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:59.279769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:59.279769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:59.279769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:59.280706 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:59.280706 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:59.280706 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:34:59.280706 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:34:59.280706 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:34:59.280706 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:34:59.280706 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:34:59.280706 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:34:59.280706 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 22:34:59.280706 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:59.280706 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 266/525] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 10 22:34:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:59.282034 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e352551e-b552-4778-852f-e885d9075b54 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:59.330338 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f49ece-4d5b-47ef-b2a9-6b966ec014f5 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:59.367100 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58665358-d363-4c68-bf30-f18d4aee9b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:59.367825 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58665358-d363-4c68-bf30-f18d4aee9b05 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:59.368102 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58665358-d363-4c68-bf30-f18d4aee9b05 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:59.368373 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58665358-d363-4c68-bf30-f18d4aee9b05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:59.375141 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:59.375499 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58665358-d363-4c68-bf30-f18d4aee9b05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:59.377047 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58665358-d363-4c68-bf30-f18d4aee9b05 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:59.386141 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e352551e-b552-4778-852f-e885d9075b54 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:59.389775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-58665358-d363-4c68-bf30-f18d4aee9b05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:34:59.389775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:59.389775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:59.389775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:59.389775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:59.389775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:59.389775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:59.389775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:59.389775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:59.389775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:59.389775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:59.389775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:59.389775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:34:59.389775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:34:59.389775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:34:59.390677 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:34:59.390677 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:59.390677 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:59.390677 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:59.390677 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:59.390677 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:59.390677 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:59.390677 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:59.390677 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:59.390677 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:59.390677 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:59.390677 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:59.390677 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:59.390677 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:59.390677 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:59.390677 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:59.390677 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:59.391514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:34:59.391514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:34:59.391514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:34:59.391514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:34:59.391514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:34:59.391514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:34:59.391514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:34:59.391514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:34:59.391514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 260/526] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:34:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:59.397053 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58665358-d363-4c68-bf30-f18d4aee9b05 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:59.427223 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51c13a-9f54-4bff-b78a-c83be055ecbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:59.427828 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51c13a-9f54-4bff-b78a-c83be055ecbf None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:59.428110 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51c13a-9f54-4bff-b78a-c83be055ecbf None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:59.428378 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51c13a-9f54-4bff-b78a-c83be055ecbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:59.440483 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a51c13a-9f54-4bff-b78a-c83be055ecbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:59.441991 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a51c13a-9f54-4bff-b78a-c83be055ecbf None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:59.458149 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-8a51c13a-9f54-4bff-b78a-c83be055ecbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:34:59.458149 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:34:59.458149 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:34:59.458149 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:34:59.458149 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:34:59.458149 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:34:59.458149 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:34:59.458149 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:34:59.458149 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:34:59.458149 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:34:59.458149 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:34:59.458149 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:34:59.458149 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:34:59.458149 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:34:59.458149 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:34:59.458149 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:34:59.459220 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:34:59.459220 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:34:59.459220 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:34:59.459220 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:34:59.459220 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:34:59.459220 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:34:59.459220 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:34:59.459220 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:34:59.459220 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:34:59.459220 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:34:59.459220 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:34:59.459220 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:34:59.459220 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:34:59.459220 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:34:59.459220 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:34:59.459220 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:34:59.459220 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:34:59.460147 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:34:59.460147 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:34:59.460147 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:34:59.461119 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 267/527] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:34:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:34:59.461769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51c13a-9f54-4bff-b78a-c83be055ecbf None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:59.517378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:59.522733 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58665358-d363-4c68-bf30-f18d4aee9b05 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:59.562714 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8387c82-29d6-4924-9c9d-8b7db405b353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:59.563791 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8387c82-29d6-4924-9c9d-8b7db405b353 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:59.564455 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8387c82-29d6-4924-9c9d-8b7db405b353 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:59.564870 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8387c82-29d6-4924-9c9d-8b7db405b353 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:59.566354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8387c82-29d6-4924-9c9d-8b7db405b353 None admin] RBAC: Adding query filter params (role.id=5dfd0f952cd641bbb2257860c4c892ac, scope.project.id=3caa2ea846f7449b8c483788fc5846c8, user.id=b22838a8ad3949e59cdbaaa1801cd07f) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:59.566878 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8387c82-29d6-4924-9c9d-8b7db405b353 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:59.569221 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8387c82-29d6-4924-9c9d-8b7db405b353 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:59.570990 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51c13a-9f54-4bff-b78a-c83be055ecbf None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:59.576617 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 261/528] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:34:59 2022] GET /identity/v3/role_assignments?user.id=b22838a8ad3949e59cdbaaa1801cd07f&scope.project.id=3caa2ea846f7449b8c483788fc5846c8&role.id=5dfd0f952cd641bbb2257860c4c892ac => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:59.577584 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8387c82-29d6-4924-9c9d-8b7db405b353 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:59.610487 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe198e3-d853-459c-af90-e33161456ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:59.611114 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe198e3-d853-459c-af90-e33161456ba2 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:59.611391 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe198e3-d853-459c-af90-e33161456ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:59.611662 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe198e3-d853-459c-af90-e33161456ba2 None admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:59.613550 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe198e3-d853-459c-af90-e33161456ba2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:59.613890 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe198e3-d853-459c-af90-e33161456ba2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:59.615392 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe198e3-d853-459c-af90-e33161456ba2 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:59.660939 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 268/529] 10.222.0.76 () {56 vars in 1139 bytes} [Wed Aug 10 22:34:59 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:59.662620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe198e3-d853-459c-af90-e33161456ba2 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:59.667093 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:59.714228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8387c82-29d6-4924-9c9d-8b7db405b353 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:59.756180 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70824c1c-5a45-41cd-9d7f-9c8fca35c89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:59.756901 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70824c1c-5a45-41cd-9d7f-9c8fca35c89b None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:59.757249 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70824c1c-5a45-41cd-9d7f-9c8fca35c89b None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:59.757633 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70824c1c-5a45-41cd-9d7f-9c8fca35c89b None admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:59.758429 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70824c1c-5a45-41cd-9d7f-9c8fca35c89b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:59.758798 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70824c1c-5a45-41cd-9d7f-9c8fca35c89b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:59.760314 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70824c1c-5a45-41cd-9d7f-9c8fca35c89b None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:59.774621 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 262/530] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:34:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:59.776321 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70824c1c-5a45-41cd-9d7f-9c8fca35c89b None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:59.803261 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe198e3-d853-459c-af90-e33161456ba2 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:59.809134 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:59.850713 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538dc9ea-a27a-457e-b068-e7da1e017ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:59.851449 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538dc9ea-a27a-457e-b068-e7da1e017ae8 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:59.851969 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538dc9ea-a27a-457e-b068-e7da1e017ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:59.852319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538dc9ea-a27a-457e-b068-e7da1e017ae8 None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:34:59.853440 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538dc9ea-a27a-457e-b068-e7da1e017ae8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:34:59.853823 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538dc9ea-a27a-457e-b068-e7da1e017ae8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:34:59.855629 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538dc9ea-a27a-457e-b068-e7da1e017ae8 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:34:59.873355 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 269/531] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:34:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:34:59.874494 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538dc9ea-a27a-457e-b068-e7da1e017ae8 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:34:59.918329 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70824c1c-5a45-41cd-9d7f-9c8fca35c89b None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:34:59.946216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:34:59.970311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de96720b-540c-40de-b2c9-e60ea00e4c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:34:59.970963 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96720b-540c-40de-b2c9-e60ea00e4c98 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:34:59.971318 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96720b-540c-40de-b2c9-e60ea00e4c98 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:34:59.971717 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96720b-540c-40de-b2c9-e60ea00e4c98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:00.000720 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538dc9ea-a27a-457e-b068-e7da1e017ae8 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:00.043103 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb9fe01-2f3b-4499-93b2-0db8ea577859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:00.044204 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9fe01-2f3b-4499-93b2-0db8ea577859 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:00.044756 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9fe01-2f3b-4499-93b2-0db8ea577859 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:00.045242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9fe01-2f3b-4499-93b2-0db8ea577859 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:00.046324 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb9fe01-2f3b-4499-93b2-0db8ea577859 None admin] RBAC: Adding query filter params (role.id=5dfd0f952cd641bbb2257860c4c892ac, scope.project.id=3caa2ea846f7449b8c483788fc5846c8, user.id=a7f0dd6c9e2d4315966e90cd3b927b13) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:00.046936 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb9fe01-2f3b-4499-93b2-0db8ea577859 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:00.048880 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb9fe01-2f3b-4499-93b2-0db8ea577859 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:00.056759 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 270/532] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:34:59 2022] GET /identity/v3/role_assignments?user.id=a7f0dd6c9e2d4315966e90cd3b927b13&scope.project.id=3caa2ea846f7449b8c483788fc5846c8&role.id=5dfd0f952cd641bbb2257860c4c892ac => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:00.057866 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb9fe01-2f3b-4499-93b2-0db8ea577859 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:00.072802 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de96720b-540c-40de-b2c9-e60ea00e4c98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:00.074624 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de96720b-540c-40de-b2c9-e60ea00e4c98 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:00.110733 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:00.210360 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 263/533] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:34:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:35:00.218759 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de96720b-540c-40de-b2c9-e60ea00e4c98 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:00.262489 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb9fe01-2f3b-4499-93b2-0db8ea577859 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:00.309078 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:00.337226 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4fa60d-f9c6-4881-adef-271c68e4c36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:00.337905 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fa60d-f9c6-4881-adef-271c68e4c36e None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:00.338186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fa60d-f9c6-4881-adef-271c68e4c36e None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:00.338459 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fa60d-f9c6-4881-adef-271c68e4c36e None admin] PATH_INFO: `/v3/projects/service` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:00.350826 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4fa60d-f9c6-4881-adef-271c68e4c36e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:00.354342 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4fa60d-f9c6-4881-adef-271c68e4c36e None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:00.359227 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de96720b-540c-40de-b2c9-e60ea00e4c98 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:00.376873 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-5a4fa60d-f9c6-4881-adef-271c68e4c36e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:00.376873 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:00.376873 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:00.376873 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:00.376873 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:00.376873 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:00.376873 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:00.376873 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:00.376873 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:00.376873 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:00.376873 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:00.376873 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:00.376873 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:35:00.376873 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:35:00.376873 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:35:00.377820 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:35:00.377820 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:00.377820 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:00.377820 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:00.377820 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:00.377820 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:00.377820 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:00.377820 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:00.377820 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:00.377820 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:00.377820 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:00.377820 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:00.377820 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:00.377820 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:00.377820 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:00.377820 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:00.377820 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:00.378731 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:35:00.378731 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:35:00.378731 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:35:00.378731 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:35:00.378731 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:35:00.378731 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:35:00.378731 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:00.378731 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:00.378731 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 271/534] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:35:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:00.383917 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4fa60d-f9c6-4881-adef-271c68e4c36e None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:00.406232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:00.435385 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c0c6a6-c0c7-417e-beae-455b7f0b3e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:00.436241 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0c6a6-c0c7-417e-beae-455b7f0b3e97 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:00.436673 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0c6a6-c0c7-417e-beae-455b7f0b3e97 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:00.437097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0c6a6-c0c7-417e-beae-455b7f0b3e97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:00.457608 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c0c6a6-c0c7-417e-beae-455b7f0b3e97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:00.462644 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c0c6a6-c0c7-417e-beae-455b7f0b3e97 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:00.500460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-75c0c6a6-c0c7-417e-beae-455b7f0b3e97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:00.500460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:00.500460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:00.500460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:00.500460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:00.500460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:00.500460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:00.500460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:00.500460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:00.500460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:00.500460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:00.500460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:00.500460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:00.500460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:00.500460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:00.500460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:00.501647 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:00.501647 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:00.501647 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:00.501647 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:00.501647 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:00.501647 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:00.501647 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:00.501647 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:00.501647 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:00.501647 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:00.501647 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:00.501647 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:00.501647 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:00.501647 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:00.501647 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:00.501647 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:00.501647 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:00.502699 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:00.502699 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:00.502699 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:00.506852 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 264/535] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:00.512886 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c0c6a6-c0c7-417e-beae-455b7f0b3e97 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:00.640046 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4fa60d-f9c6-4881-adef-271c68e4c36e None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:00.641731 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c0c6a6-c0c7-417e-beae-455b7f0b3e97 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:00.681979 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b1730-1291-4a0e-b61a-a700fa856f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:00.682572 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b1730-1291-4a0e-b61a-a700fa856f0b None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:00.682847 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b1730-1291-4a0e-b61a-a700fa856f0b None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:00.683117 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b1730-1291-4a0e-b61a-a700fa856f0b None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:00.683754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b1730-1291-4a0e-b61a-a700fa856f0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:00.684049 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b1730-1291-4a0e-b61a-a700fa856f0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:00.685531 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b1730-1291-4a0e-b61a-a700fa856f0b None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:00.697979 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197473db-e35f-4ef1-bc00-cc475241b1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:00.698587 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197473db-e35f-4ef1-bc00-cc475241b1c1 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:00.699133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197473db-e35f-4ef1-bc00-cc475241b1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:00.699404 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197473db-e35f-4ef1-bc00-cc475241b1c1 None admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:00.700175 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197473db-e35f-4ef1-bc00-cc475241b1c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:00.700482 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197473db-e35f-4ef1-bc00-cc475241b1c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:00.702020 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197473db-e35f-4ef1-bc00-cc475241b1c1 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:00.703202 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 265/536] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:00.704525 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666be5a-0f22-44d8-a87f-915ee5b2887e None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:00.704811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666be5a-0f22-44d8-a87f-915ee5b2887e None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:00.705086 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666be5a-0f22-44d8-a87f-915ee5b2887e None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:00.707248 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 266/537] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:00.707947 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666be5a-0f22-44d8-a87f-915ee5b2887e None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:00.727888 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 272/538] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:35:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:35:00.729451 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b604b-b4b5-46b9-a1ab-9e7a0ac0681d None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:00.729734 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b604b-b4b5-46b9-a1ab-9e7a0ac0681d None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:00.730775 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b604b-b4b5-46b9-a1ab-9e7a0ac0681d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:00.748035 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:00.752216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-ac8b604b-b4b5-46b9-a1ab-9e7a0ac0681d None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:00.754092 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-ac8b604b-b4b5-46b9-a1ab-9e7a0ac0681d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:00.803247 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:00.820496 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666be5a-0f22-44d8-a87f-915ee5b2887e None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:00.832825 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 273/539] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:35:00.833777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b604b-b4b5-46b9-a1ab-9e7a0ac0681d None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:00.857700 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c262b-7c8b-4607-8343-6d6b5da3fee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:00.858669 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c262b-7c8b-4607-8343-6d6b5da3fee8 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:00.859200 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c262b-7c8b-4607-8343-6d6b5da3fee8 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:00.859695 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c262b-7c8b-4607-8343-6d6b5da3fee8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:00.866951 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c262b-7c8b-4607-8343-6d6b5da3fee8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:00.868664 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c262b-7c8b-4607-8343-6d6b5da3fee8 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:00.880774 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-6c4c262b-7c8b-4607-8343-6d6b5da3fee8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:00.880774 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:00.880774 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:00.880774 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:00.880774 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:00.880774 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:00.880774 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:00.880774 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:00.880774 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:00.880774 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:00.880774 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:00.880774 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:00.880774 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:35:00.880774 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:35:00.880774 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:35:00.881697 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:35:00.881697 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:00.881697 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:00.881697 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:00.881697 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:00.881697 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:00.881697 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:00.881697 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:00.881697 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:00.881697 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:00.881697 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:00.881697 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:00.881697 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:00.881697 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:00.881697 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:00.881697 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:00.881697 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:00.882524 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:35:00.882524 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:35:00.882524 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:35:00.882524 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:35:00.882524 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:35:00.882524 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:35:00.882524 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:00.882524 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:00.884849 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 267/540] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:35:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:00.885862 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c262b-7c8b-4607-8343-6d6b5da3fee8 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:00.951396 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b604b-b4b5-46b9-a1ab-9e7a0ac0681d None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:00.984350 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:00.992889 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c262b-7c8b-4607-8343-6d6b5da3fee8 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:00.994026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6665143-120a-4ef6-a4cd-32744139aaf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:00.995187 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6665143-120a-4ef6-a4cd-32744139aaf5 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:00.995673 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6665143-120a-4ef6-a4cd-32744139aaf5 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:00.996800 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6665143-120a-4ef6-a4cd-32744139aaf5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:00.997493 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6665143-120a-4ef6-a4cd-32744139aaf5 None admin] RBAC: Adding query filter params (role.id=5dfd0f952cd641bbb2257860c4c892ac, scope.project.id=3caa2ea846f7449b8c483788fc5846c8, user.id=dc7524edf5d5422f91d92fd91be5ee1e) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:00.997958 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6665143-120a-4ef6-a4cd-32744139aaf5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:00.999403 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6665143-120a-4ef6-a4cd-32744139aaf5 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:01.013481 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 274/541] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:35:00 2022] GET /identity/v3/role_assignments?user.id=dc7524edf5d5422f91d92fd91be5ee1e&scope.project.id=3caa2ea846f7449b8c483788fc5846c8&role.id=5dfd0f952cd641bbb2257860c4c892ac => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:01.014508 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6665143-120a-4ef6-a4cd-32744139aaf5 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:01.033046 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e947aa-c92f-4a60-a91d-8349272581b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:01.036367 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e947aa-c92f-4a60-a91d-8349272581b6 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:01.037640 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e947aa-c92f-4a60-a91d-8349272581b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:01.038322 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e947aa-c92f-4a60-a91d-8349272581b6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:01.063195 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e947aa-c92f-4a60-a91d-8349272581b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:35:01.073432 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-52e947aa-c92f-4a60-a91d-8349272581b6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:35:01.073432 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:01.073432 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:01.073432 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:01.073432 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:01.073432 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:01.073432 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:01.073432 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:01.073432 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:01.073432 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:01.073432 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:01.073432 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:01.073432 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:35:01.073432 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:35:01.073432 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:35:01.073432 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:35:01.074649 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:35:01.074649 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:35:01.074649 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:35:01.074649 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:35:01.074649 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:01.074649 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:01.074649 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:01.074649 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:01.074649 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:01.074649 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:01.074649 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:01.074649 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:01.074649 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:01.074649 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:01.074649 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:01.074649 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:01.075422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:01.075422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:01.075422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:01.075422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:01.075422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:35:01.075422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:35:01.075422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:35:01.075422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:35:01.075422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:35:01.075422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:35:01.075422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:35:01.075422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:01.078679 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 268/542] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 10 22:35:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:01.086031 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e947aa-c92f-4a60-a91d-8349272581b6 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:01.104636 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:01.233717 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6665143-120a-4ef6-a4cd-32744139aaf5 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:01.241831 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e947aa-c92f-4a60-a91d-8349272581b6 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:01.274248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f3b42-1845-4aca-b812-3a0299db04c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:01.275322 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f3b42-1845-4aca-b812-3a0299db04c5 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:01.275826 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f3b42-1845-4aca-b812-3a0299db04c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:01.276311 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f3b42-1845-4aca-b812-3a0299db04c5 None admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:01.277321 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f3b42-1845-4aca-b812-3a0299db04c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:01.277873 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f3b42-1845-4aca-b812-3a0299db04c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:01.279466 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2c0e95-0e53-4d43-9750-817779a4468c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:01.279860 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f3b42-1845-4aca-b812-3a0299db04c5 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:01.280097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c0e95-0e53-4d43-9750-817779a4468c None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:01.280376 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c0e95-0e53-4d43-9750-817779a4468c None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:01.280646 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c0e95-0e53-4d43-9750-817779a4468c None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:01.281352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2c0e95-0e53-4d43-9750-817779a4468c None admin] RBAC: Adding query filter params (name=service) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:01.281657 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2c0e95-0e53-4d43-9750-817779a4468c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:01.283155 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2c0e95-0e53-4d43-9750-817779a4468c None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:01.292706 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 269/543] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 10 22:35:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:01.297606 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 275/544] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:35:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:01.298547 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2c0e95-0e53-4d43-9750-817779a4468c None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:01.306316 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f3b42-1845-4aca-b812-3a0299db04c5 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:01.336118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:01.402972 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2c0e95-0e53-4d43-9750-817779a4468c None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:01.457822 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f179b-cf31-4e4d-8718-e2f34f119f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:01.458461 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f179b-cf31-4e4d-8718-e2f34f119f1e None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:01.458736 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f179b-cf31-4e4d-8718-e2f34f119f1e None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:01.459001 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f179b-cf31-4e4d-8718-e2f34f119f1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:01.478089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514f179b-cf31-4e4d-8718-e2f34f119f1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:01.479538 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514f179b-cf31-4e4d-8718-e2f34f119f1e None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:01.494357 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:01.497086 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f3b42-1845-4aca-b812-3a0299db04c5 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:01.510445 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-514f179b-cf31-4e4d-8718-e2f34f119f1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:01.510445 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:01.510445 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:01.510445 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:01.510445 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:01.510445 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:01.510445 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:01.510445 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:01.510445 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:01.510445 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:01.510445 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:01.510445 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:01.510445 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:01.510445 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:01.510445 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:01.510445 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:01.511378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:01.511378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:01.511378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:01.511378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:01.511378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:01.511378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:01.511378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:01.511378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:01.511378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:01.511378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:01.511378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:01.511378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:01.511378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:01.511378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:01.511378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:01.511378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:01.511378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:01.512826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:01.512826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:01.512826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:01.512826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 270/545] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:35:01.517158 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f179b-cf31-4e4d-8718-e2f34f119f1e None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:01.553741 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9948a40-6939-429f-8cc3-d399da55de55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:01.554404 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9948a40-6939-429f-8cc3-d399da55de55 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:01.554681 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9948a40-6939-429f-8cc3-d399da55de55 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:01.554948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9948a40-6939-429f-8cc3-d399da55de55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:01.555673 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9948a40-6939-429f-8cc3-d399da55de55 None admin] RBAC: Adding query filter params (role.id=5dfd0f952cd641bbb2257860c4c892ac, scope.project.id=3caa2ea846f7449b8c483788fc5846c8, user.id=7fcb5a674ff94b3d8f79efbbe4959069) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:01.555975 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9948a40-6939-429f-8cc3-d399da55de55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:01.557455 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9948a40-6939-429f-8cc3-d399da55de55 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:01.570399 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 276/546] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:35:01 2022] GET /identity/v3/role_assignments?user.id=7fcb5a674ff94b3d8f79efbbe4959069&scope.project.id=3caa2ea846f7449b8c483788fc5846c8&role.id=5dfd0f952cd641bbb2257860c4c892ac => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:01.641916 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:01.689796 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f179b-cf31-4e4d-8718-e2f34f119f1e None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:01.733306 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf017488-294f-4025-9d55-431e1ecb6e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:01.734317 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf017488-294f-4025-9d55-431e1ecb6e7f None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:01.734825 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf017488-294f-4025-9d55-431e1ecb6e7f None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:01.735417 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf017488-294f-4025-9d55-431e1ecb6e7f None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:01.736297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf017488-294f-4025-9d55-431e1ecb6e7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:01.736810 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf017488-294f-4025-9d55-431e1ecb6e7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:01.738877 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf017488-294f-4025-9d55-431e1ecb6e7f None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:01.752687 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 271/547] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:01.758131 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9948a40-6939-429f-8cc3-d399da55de55 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:01.877296 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9948a40-6939-429f-8cc3-d399da55de55 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:01.914536 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21a5425-c09d-4d27-90f0-e4c4cb44ee62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:01.915502 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a5425-c09d-4d27-90f0-e4c4cb44ee62 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:01.915979 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a5425-c09d-4d27-90f0-e4c4cb44ee62 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:01.916512 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a5425-c09d-4d27-90f0-e4c4cb44ee62 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:01.924886 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21a5425-c09d-4d27-90f0-e4c4cb44ee62 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:01.927325 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21a5425-c09d-4d27-90f0-e4c4cb44ee62 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:01.939572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-e21a5425-c09d-4d27-90f0-e4c4cb44ee62 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 22:35:01.939572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:01.939572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:01.939572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:01.939572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:01.939572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:01.939572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:01.939572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:01.939572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:01.939572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:01.939572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:01.939572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:01.939572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:35:01.939572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:35:01.939572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:35:01.939572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:35:01.940631 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:01.940631 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:01.940631 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:35:01.940631 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:01.940631 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:35:01.940631 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:01.940631 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:01.940631 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:01.940631 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:01.940631 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:01.940631 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:01.940631 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:01.940631 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:01.940631 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:01.940631 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:01.940631 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:01.940631 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:01.940631 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:01.942660 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:01.942660 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:01.942660 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:35:01.942660 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:35:01.942660 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:35:01.942660 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:35:01.942660 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:35:01.942660 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:35:01.942660 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 22:35:01.942660 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:01.945641 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 277/548] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:35:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:35:01.954375 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf017488-294f-4025-9d55-431e1ecb6e7f None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:01.968531 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fedea04-92f5-4749-9a4e-eeba909290bf None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:01.968835 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fedea04-92f5-4749-9a4e-eeba909290bf None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:01.969075 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fedea04-92f5-4749-9a4e-eeba909290bf None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:01.970678 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 278/549] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:35:01.975786 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e317306-54d2-4461-b146-9481bb29893d None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:01.976185 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e317306-54d2-4461-b146-9481bb29893d None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:01.976463 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e317306-54d2-4461-b146-9481bb29893d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:01.996092 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-1e317306-54d2-4461-b146-9481bb29893d None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:01.997325 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-1e317306-54d2-4461-b146-9481bb29893d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:02.030501 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:02.117404 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 279/550] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:35:02.133088 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf017488-294f-4025-9d55-431e1ecb6e7f None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:02.160723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e317306-54d2-4461-b146-9481bb29893d None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:02.190042 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:02.204395 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a262e1b-5f50-4aaa-badf-539cff01229c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:02.205009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a262e1b-5f50-4aaa-badf-539cff01229c None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:02.205323 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a262e1b-5f50-4aaa-badf-539cff01229c None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:02.205621 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a262e1b-5f50-4aaa-badf-539cff01229c None admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:02.206383 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a262e1b-5f50-4aaa-badf-539cff01229c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:02.207222 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a262e1b-5f50-4aaa-badf-539cff01229c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:02.208659 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a262e1b-5f50-4aaa-badf-539cff01229c None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:02.227533 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 272/551] 10.222.0.76 () {56 vars in 1135 bytes} [Wed Aug 10 22:35:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:02.232642 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a262e1b-5f50-4aaa-badf-539cff01229c None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:02.285948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e317306-54d2-4461-b146-9481bb29893d None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:02.326403 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f57f2f2-1b03-4d44-958f-483ce129df36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:02.327432 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57f2f2-1b03-4d44-958f-483ce129df36 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:02.327920 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57f2f2-1b03-4d44-958f-483ce129df36 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:02.328394 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57f2f2-1b03-4d44-958f-483ce129df36 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:02.364923 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f57f2f2-1b03-4d44-958f-483ce129df36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:35:02.369689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-5f57f2f2-1b03-4d44-958f-483ce129df36 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:35:02.369689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:02.369689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:02.369689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:02.369689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:02.369689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:02.369689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:02.369689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:02.369689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:02.369689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:02.369689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:02.369689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:02.369689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:35:02.369689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:35:02.369689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:35:02.369689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:35:02.370683 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:35:02.370683 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:35:02.370683 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:35:02.370683 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:35:02.370683 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:02.370683 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:02.370683 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:02.370683 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:02.370683 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:02.370683 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:02.370683 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:02.370683 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:02.370683 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:02.370683 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:02.370683 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:02.370683 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:02.372036 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:02.372036 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:02.372036 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:02.372036 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:02.372036 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:35:02.372036 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:35:02.372036 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:35:02.372036 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:35:02.372036 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:35:02.372036 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:35:02.372036 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:35:02.372036 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:02.372036 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 280/552] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 10 22:35:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:02.378847 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f57f2f2-1b03-4d44-958f-483ce129df36 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:02.398535 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:02.404045 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a262e1b-5f50-4aaa-badf-539cff01229c None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:02.441391 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03222222-fe89-4922-b238-985a3fe4e610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:02.441996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03222222-fe89-4922-b238-985a3fe4e610 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:02.442278 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03222222-fe89-4922-b238-985a3fe4e610 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:02.442545 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03222222-fe89-4922-b238-985a3fe4e610 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:02.499425 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f57f2f2-1b03-4d44-958f-483ce129df36 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:02.505172 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:02.520868 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03222222-fe89-4922-b238-985a3fe4e610 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:02.522407 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03222222-fe89-4922-b238-985a3fe4e610 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:02.542409 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129caca8-e7b0-4a3b-b3c0-55b0e41cfc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:02.543408 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129caca8-e7b0-4a3b-b3c0-55b0e41cfc27 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:02.543829 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129caca8-e7b0-4a3b-b3c0-55b0e41cfc27 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:02.544241 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129caca8-e7b0-4a3b-b3c0-55b0e41cfc27 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:02.545050 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129caca8-e7b0-4a3b-b3c0-55b0e41cfc27 None admin] RBAC: Adding query filter params (name=service) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:02.545544 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129caca8-e7b0-4a3b-b3c0-55b0e41cfc27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:02.547169 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129caca8-e7b0-4a3b-b3c0-55b0e41cfc27 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:02.561550 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 281/553] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 10 22:35:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:02.563197 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f725e1b-3988-427f-9c04-1e57cdd8b8db None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:02.563628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f725e1b-3988-427f-9c04-1e57cdd8b8db None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:02.564069 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f725e1b-3988-427f-9c04-1e57cdd8b8db None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:02.565911 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 282/554] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:02.567795 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f725e1b-3988-427f-9c04-1e57cdd8b8db None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:02.655352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 273/555] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:35:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 22:35:02.655352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f5ba4-525d-4f07-93ee-319131d4462b None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:02.655352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f5ba4-525d-4f07-93ee-319131d4462b None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:02.655352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f5ba4-525d-4f07-93ee-319131d4462b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:02.677947 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-a76f5ba4-525d-4f07-93ee-319131d4462b None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:02.679146 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-a76f5ba4-525d-4f07-93ee-319131d4462b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:02.687807 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:02.716130 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f725e1b-3988-427f-9c04-1e57cdd8b8db None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:02.753386 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4212ed7-f83a-47bb-ab4f-f501601f7d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:02.754538 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4212ed7-f83a-47bb-ab4f-f501601f7d6f None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:02.755029 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4212ed7-f83a-47bb-ab4f-f501601f7d6f None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:02.755559 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4212ed7-f83a-47bb-ab4f-f501601f7d6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:02.757534 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 274/556] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:35:02.758458 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f5ba4-525d-4f07-93ee-319131d4462b None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:02.770053 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4212ed7-f83a-47bb-ab4f-f501601f7d6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:02.772242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4212ed7-f83a-47bb-ab4f-f501601f7d6f None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:02.788672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-d4212ed7-f83a-47bb-ab4f-f501601f7d6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:02.788672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:02.788672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:02.788672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:02.788672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:02.788672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:02.788672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:02.788672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:02.788672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:02.788672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:02.788672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:02.788672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:02.788672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:02.788672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:02.788672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:02.788672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:02.789711 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:02.789711 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:02.789711 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:02.789711 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:02.789711 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:02.789711 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:02.789711 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:02.789711 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:02.789711 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:02.789711 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:02.789711 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:02.789711 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:02.789711 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:02.789711 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:02.789711 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:02.789711 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:02.789711 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:02.790567 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:02.790567 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:02.790567 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:02.790748 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:02.792995 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 283/557] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:35:02.804168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4212ed7-f83a-47bb-ab4f-f501601f7d6f None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:02.885796 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f5ba4-525d-4f07-93ee-319131d4462b None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:02.918304 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4212ed7-f83a-47bb-ab4f-f501601f7d6f None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:02.923325 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90259e3d-4e8c-4489-92a1-f325cfc98271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:02.923978 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90259e3d-4e8c-4489-92a1-f325cfc98271 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:02.924257 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90259e3d-4e8c-4489-92a1-f325cfc98271 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:02.924527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90259e3d-4e8c-4489-92a1-f325cfc98271 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:02.937954 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90259e3d-4e8c-4489-92a1-f325cfc98271 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:02.940067 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90259e3d-4e8c-4489-92a1-f325cfc98271 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:02.951274 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:02.960892 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-90259e3d-4e8c-4489-92a1-f325cfc98271 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:02.960892 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:02.960892 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:02.960892 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:02.960892 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:02.960892 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:02.960892 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:02.960892 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:02.960892 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:02.960892 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:02.960892 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:02.960892 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:02.960892 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:02.960892 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:02.960892 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:02.960892 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:02.962386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:02.962386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:02.962386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:02.962386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:02.962386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:02.962386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:02.962386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:02.962386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:02.962386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:02.962386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:02.962386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:02.962386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:02.962386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:02.962386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:02.962386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:02.962386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:02.962386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:02.963256 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:02.963256 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:02.963256 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:02.965578 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 275/558] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:02.966322 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90259e3d-4e8c-4489-92a1-f325cfc98271 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:02.967791 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1e404-b946-4060-93c3-08dd5efe4e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:02.968681 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1e404-b946-4060-93c3-08dd5efe4e7d None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:02.969199 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1e404-b946-4060-93c3-08dd5efe4e7d None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:02.969683 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1e404-b946-4060-93c3-08dd5efe4e7d None admin] PATH_INFO: `/v3/roles/service` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:02.983276 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd1e404-b946-4060-93c3-08dd5efe4e7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:35:02.989785 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-5bd1e404-b946-4060-93c3-08dd5efe4e7d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:35:02.989785 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:02.989785 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:02.989785 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:02.989785 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:02.989785 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:02.989785 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:02.989785 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:02.989785 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:02.989785 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:02.989785 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:02.989785 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:02.989785 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:35:02.989785 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:35:02.989785 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:35:02.989785 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:35:02.990727 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:35:02.990727 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:35:02.990727 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:35:02.990727 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:35:02.990727 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:02.990727 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:02.990727 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:02.990727 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:02.990727 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:02.990727 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:02.990727 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:02.990727 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:02.990727 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:02.990727 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:02.990727 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:02.990727 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:02.991557 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:02.991557 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:02.991557 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:02.991557 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:02.991557 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:35:02.991557 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:35:02.991557 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:35:02.991557 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:35:02.991557 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:35:02.991557 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:35:02.991557 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:35:02.991557 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:02.998874 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 284/559] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 10 22:35:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:35:03.000122 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1e404-b946-4060-93c3-08dd5efe4e7d None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:03.057681 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:03.106454 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90259e3d-4e8c-4489-92a1-f325cfc98271 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:03.116240 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1e404-b946-4060-93c3-08dd5efe4e7d None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:03.148540 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66835a8f-ba05-4d89-b71e-5f270b5e6739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:03.149458 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66835a8f-ba05-4d89-b71e-5f270b5e6739 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:03.149741 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66835a8f-ba05-4d89-b71e-5f270b5e6739 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:03.150019 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66835a8f-ba05-4d89-b71e-5f270b5e6739 None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:03.150661 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66835a8f-ba05-4d89-b71e-5f270b5e6739 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:03.150962 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66835a8f-ba05-4d89-b71e-5f270b5e6739 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:03.152795 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66835a8f-ba05-4d89-b71e-5f270b5e6739 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:03.157492 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ac744-60e3-4bf0-a295-3528b270ec23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:03.158359 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ac744-60e3-4bf0-a295-3528b270ec23 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:03.158856 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ac744-60e3-4bf0-a295-3528b270ec23 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:03.159348 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ac744-60e3-4bf0-a295-3528b270ec23 None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:03.160856 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ac744-60e3-4bf0-a295-3528b270ec23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:03.161428 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ac744-60e3-4bf0-a295-3528b270ec23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:03.163094 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ac744-60e3-4bf0-a295-3528b270ec23 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:03.173416 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 276/560] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:03.173416 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66835a8f-ba05-4d89-b71e-5f270b5e6739 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:03.191321 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 285/561] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:03.191321 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ac744-60e3-4bf0-a295-3528b270ec23 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:03.244451 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:03.305151 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ac744-60e3-4bf0-a295-3528b270ec23 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:03.314180 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66835a8f-ba05-4d89-b71e-5f270b5e6739 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:03.346001 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d0656a-7e94-4e0a-8a39-095e5da63bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:03.346838 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0656a-7e94-4e0a-8a39-095e5da63bd4 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:03.347881 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0656a-7e94-4e0a-8a39-095e5da63bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:03.348443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0656a-7e94-4e0a-8a39-095e5da63bd4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:03.354576 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:03.357796 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d0656a-7e94-4e0a-8a39-095e5da63bd4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:03.359777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d0656a-7e94-4e0a-8a39-095e5da63bd4 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:03.366125 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab03e1c-8266-4dbe-8332-2bdca9c11917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:03.367035 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab03e1c-8266-4dbe-8332-2bdca9c11917 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:03.367510 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab03e1c-8266-4dbe-8332-2bdca9c11917 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:03.368022 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab03e1c-8266-4dbe-8332-2bdca9c11917 None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:03.369145 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab03e1c-8266-4dbe-8332-2bdca9c11917 None admin] RBAC: Adding query filter params (name=service) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:03.369826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab03e1c-8266-4dbe-8332-2bdca9c11917 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:03.371407 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab03e1c-8266-4dbe-8332-2bdca9c11917 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:03.375378 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-10d0656a-7e94-4e0a-8a39-095e5da63bd4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:35:03.375378 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:03.375378 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:03.375378 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:03.375378 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:03.375378 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:03.375378 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:03.375378 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:03.375378 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:03.375378 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:03.375378 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:03.375378 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:03.375378 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:35:03.375378 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:35:03.375378 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:35:03.375378 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:35:03.376458 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:03.376458 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:03.376458 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:35:03.376458 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:03.376458 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:35:03.376458 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:03.376458 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:03.376458 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:03.376458 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:03.376458 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:03.376458 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:03.376458 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:03.376458 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:03.376458 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:03.376458 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:03.376458 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:03.376458 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:03.376458 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:03.377656 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:03.377656 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:03.377656 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:35:03.377656 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:35:03.377656 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:35:03.377656 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:35:03.377656 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:35:03.377656 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:35:03.377656 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:35:03.377656 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:03.384338 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 277/562] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 10 22:35:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:03.384795 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 286/563] 10.222.0.76 () {56 vars in 1093 bytes} [Wed Aug 10 22:35:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:03.385606 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab03e1c-8266-4dbe-8332-2bdca9c11917 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:03.390298 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d0656a-7e94-4e0a-8a39-095e5da63bd4 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:03.503013 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d0656a-7e94-4e0a-8a39-095e5da63bd4 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:03.512564 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:03.520913 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab03e1c-8266-4dbe-8332-2bdca9c11917 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:03.540664 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b84df99-68f4-4e71-adaf-c9c7e500a26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:03.541306 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84df99-68f4-4e71-adaf-c9c7e500a26c None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:03.541597 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84df99-68f4-4e71-adaf-c9c7e500a26c None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:03.541879 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84df99-68f4-4e71-adaf-c9c7e500a26c None admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:03.542605 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b84df99-68f4-4e71-adaf-c9c7e500a26c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:03.542915 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b84df99-68f4-4e71-adaf-c9c7e500a26c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:03.544358 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b84df99-68f4-4e71-adaf-c9c7e500a26c None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:03.558388 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cc5128-3114-429b-afb1-4797d920c824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:03.559224 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc5128-3114-429b-afb1-4797d920c824 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:03.559779 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc5128-3114-429b-afb1-4797d920c824 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:03.560052 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc5128-3114-429b-afb1-4797d920c824 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:03.567928 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 287/564] 10.222.0.76 () {56 vars in 1133 bytes} [Wed Aug 10 22:35:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:03.567928 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b84df99-68f4-4e71-adaf-c9c7e500a26c None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:03.569951 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cc5128-3114-429b-afb1-4797d920c824 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:03.571898 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cc5128-3114-429b-afb1-4797d920c824 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:03.584657 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-52cc5128-3114-429b-afb1-4797d920c824 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:03.584657 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:03.584657 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:03.584657 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:03.584657 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:03.584657 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:03.584657 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:03.584657 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:03.584657 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:03.584657 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:03.584657 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:03.584657 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:03.584657 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:35:03.584657 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:35:03.584657 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:35:03.585742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:35:03.585742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:03.585742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:03.585742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:03.585742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:03.585742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:03.585742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:03.585742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:03.585742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:03.585742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:03.585742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:03.585742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:03.585742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:03.585742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:03.585742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:03.585742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:03.585742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:03.586779 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:35:03.586779 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:35:03.586779 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:35:03.586779 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:35:03.586779 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:35:03.586779 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:35:03.586779 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:03.586779 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:03.586779 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 278/565] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:35:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:03.587331 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff31e7-667f-4083-ba82-79423dbb99c4 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:03.587619 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff31e7-667f-4083-ba82-79423dbb99c4 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:03.588107 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff31e7-667f-4083-ba82-79423dbb99c4 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:03.593253 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 279/566] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:03.593253 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ff31e7-667f-4083-ba82-79423dbb99c4 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:03.648847 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:03.681518 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b84df99-68f4-4e71-adaf-c9c7e500a26c None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:03.704183 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ff31e7-667f-4083-ba82-79423dbb99c4 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:03.718771 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27769635-e051-4fb9-8cd6-7a5a47555835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:03.719400 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27769635-e051-4fb9-8cd6-7a5a47555835 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:03.719673 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27769635-e051-4fb9-8cd6-7a5a47555835 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:03.719940 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27769635-e051-4fb9-8cd6-7a5a47555835 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:03.730837 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27769635-e051-4fb9-8cd6-7a5a47555835 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:03.732289 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27769635-e051-4fb9-8cd6-7a5a47555835 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:03.742401 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248aef0a-e304-48b0-83cf-2cee38482d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:03.743102 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248aef0a-e304-48b0-83cf-2cee38482d7f None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:03.743451 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248aef0a-e304-48b0-83cf-2cee38482d7f None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:03.743792 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248aef0a-e304-48b0-83cf-2cee38482d7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:03.747259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-27769635-e051-4fb9-8cd6-7a5a47555835 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:03.747259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:03.747259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:03.747259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:03.747259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:03.747259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:03.747259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:03.747259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:03.747259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:03.747259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:03.747259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:03.747259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:03.747259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:03.747259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:03.747259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:03.747259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:03.748377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:03.748377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:03.748377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:03.748377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:03.748377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:03.748377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:03.748377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:03.748377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:03.748377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:03.748377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:03.748377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:03.748377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:03.748377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:03.748377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:03.748377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:03.748377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:03.748377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:03.749330 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:03.749330 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:03.749330 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:03.749330 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 288/567] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:03.749561 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27769635-e051-4fb9-8cd6-7a5a47555835 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:03.760986 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248aef0a-e304-48b0-83cf-2cee38482d7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:03.762552 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248aef0a-e304-48b0-83cf-2cee38482d7f None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:03.770230 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:03.779027 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-248aef0a-e304-48b0-83cf-2cee38482d7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:03.779027 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:03.779027 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:03.779027 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:03.779027 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:03.779027 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:03.779027 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:03.779027 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:03.779027 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:03.779027 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:03.779027 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:03.779027 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:03.779027 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:03.779027 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:03.779027 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:03.779027 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:03.780161 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:03.780161 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:03.780161 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:03.780161 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:03.780161 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:03.780161 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:03.780161 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:03.780161 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:03.780161 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:03.780161 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:03.780161 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:03.780161 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:03.780161 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:03.780161 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:03.780161 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:03.780161 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:03.780161 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:03.781066 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:03.781066 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:03.781066 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:03.781066 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 280/568] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:03.781635 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc6a86-d6d7-4c85-ba65-5e66216c5d59 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:03.782015 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc6a86-d6d7-4c85-ba65-5e66216c5d59 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:03.782291 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc6a86-d6d7-4c85-ba65-5e66216c5d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:03.801124 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-b1dc6a86-d6d7-4c85-ba65-5e66216c5d59 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:03.802528 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-b1dc6a86-d6d7-4c85-ba65-5e66216c5d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:03.871979 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27769635-e051-4fb9-8cd6-7a5a47555835 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:03.877495 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 281/569] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:35:03.877495 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc6a86-d6d7-4c85-ba65-5e66216c5d59 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:03.913781 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b83a46-a95f-4150-a0c9-8fa59b5aefac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:03.914397 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b83a46-a95f-4150-a0c9-8fa59b5aefac None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:03.914685 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b83a46-a95f-4150-a0c9-8fa59b5aefac None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:03.914942 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b83a46-a95f-4150-a0c9-8fa59b5aefac None admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:03.915720 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b83a46-a95f-4150-a0c9-8fa59b5aefac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:03.916086 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b83a46-a95f-4150-a0c9-8fa59b5aefac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:03.917596 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b83a46-a95f-4150-a0c9-8fa59b5aefac None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:03.921044 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:03.930538 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 289/570] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:35:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:03.931274 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b83a46-a95f-4150-a0c9-8fa59b5aefac None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:03.998729 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc6a86-d6d7-4c85-ba65-5e66216c5d59 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:04.011429 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:04.033446 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a5bb8-54d3-4407-a63e-29f344ad1f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:04.033660 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b83a46-a95f-4150-a0c9-8fa59b5aefac None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:04.034050 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a5bb8-54d3-4407-a63e-29f344ad1f24 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:04.034325 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a5bb8-54d3-4407-a63e-29f344ad1f24 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:04.034595 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a5bb8-54d3-4407-a63e-29f344ad1f24 None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:04.035368 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a5bb8-54d3-4407-a63e-29f344ad1f24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:04.035665 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a5bb8-54d3-4407-a63e-29f344ad1f24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:04.037033 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a5bb8-54d3-4407-a63e-29f344ad1f24 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:04.047976 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 282/571] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:04.048760 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a5bb8-54d3-4407-a63e-29f344ad1f24 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:04.074613 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff63c04-408f-487d-bd17-23b950540f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:04.075227 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff63c04-408f-487d-bd17-23b950540f5e None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:04.075508 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff63c04-408f-487d-bd17-23b950540f5e None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:04.075782 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff63c04-408f-487d-bd17-23b950540f5e None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:04.076435 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff63c04-408f-487d-bd17-23b950540f5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:04.076715 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff63c04-408f-487d-bd17-23b950540f5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:04.078130 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff63c04-408f-487d-bd17-23b950540f5e None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:04.089994 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 290/572] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:04.090827 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff63c04-408f-487d-bd17-23b950540f5e None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:04.143645 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:04.152857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a5bb8-54d3-4407-a63e-29f344ad1f24 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:04.186475 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44cf8a4-b42b-45ed-9277-79776ee76d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:04.187105 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44cf8a4-b42b-45ed-9277-79776ee76d59 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:04.187383 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44cf8a4-b42b-45ed-9277-79776ee76d59 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:04.187651 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44cf8a4-b42b-45ed-9277-79776ee76d59 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:04.194793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff63c04-408f-487d-bd17-23b950540f5e None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:04.198450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44cf8a4-b42b-45ed-9277-79776ee76d59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:35:04.203902 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-c44cf8a4-b42b-45ed-9277-79776ee76d59 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:35:04.203902 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:04.203902 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:04.203902 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:04.203902 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:04.203902 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:04.203902 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:04.203902 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:04.203902 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:04.203902 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:04.203902 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:04.203902 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:04.203902 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:35:04.203902 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:35:04.203902 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:35:04.203902 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:35:04.204996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:35:04.204996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:35:04.204996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:35:04.204996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:35:04.204996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:04.204996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:04.204996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:04.204996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:04.204996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:04.204996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:04.204996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:04.204996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:04.204996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:04.204996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:04.204996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:04.204996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:04.205885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:04.205885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:04.205885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:04.205885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:04.205885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:35:04.205885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:35:04.205885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:35:04.205885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:35:04.205885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:35:04.205885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:35:04.205885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:35:04.205885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:04.205885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 283/573] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 10 22:35:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:04.206772 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f1fdc2-ee8e-4721-8773-d506c1acbfef None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:04.207235 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f1fdc2-ee8e-4721-8773-d506c1acbfef None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:04.207500 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f1fdc2-ee8e-4721-8773-d506c1acbfef None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:04.208749 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 284/574] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:04.209899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f1fdc2-ee8e-4721-8773-d506c1acbfef None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:04.236721 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47caeb-8414-4e36-80ac-e4ca24a98e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:04.237382 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47caeb-8414-4e36-80ac-e4ca24a98e72 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:04.237663 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47caeb-8414-4e36-80ac-e4ca24a98e72 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:04.238577 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47caeb-8414-4e36-80ac-e4ca24a98e72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:04.239405 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47caeb-8414-4e36-80ac-e4ca24a98e72 None admin] RBAC: Adding query filter params (role.id=5dfd0f952cd641bbb2257860c4c892ac, scope.project.id=3caa2ea846f7449b8c483788fc5846c8, user.id=1e3a76df95a34b9fb9816f22b85d1071) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:04.239707 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47caeb-8414-4e36-80ac-e4ca24a98e72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:04.241183 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47caeb-8414-4e36-80ac-e4ca24a98e72 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:04.246614 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 291/575] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:35:04 2022] GET /identity/v3/role_assignments?user.id=1e3a76df95a34b9fb9816f22b85d1071&scope.project.id=3caa2ea846f7449b8c483788fc5846c8&role.id=5dfd0f952cd641bbb2257860c4c892ac => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:04.247385 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47caeb-8414-4e36-80ac-e4ca24a98e72 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:04.249914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:04.325989 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f1fdc2-ee8e-4721-8773-d506c1acbfef None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:04.368014 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5b12f-29c1-4d9a-a9d0-3df21ec2f628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:04.369414 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5b12f-29c1-4d9a-a9d0-3df21ec2f628 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:04.370165 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5b12f-29c1-4d9a-a9d0-3df21ec2f628 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:04.371044 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5b12f-29c1-4d9a-a9d0-3df21ec2f628 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:04.373765 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47caeb-8414-4e36-80ac-e4ca24a98e72 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:04.381737 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc5b12f-29c1-4d9a-a9d0-3df21ec2f628 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:04.383287 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc5b12f-29c1-4d9a-a9d0-3df21ec2f628 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:04.396460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-3bc5b12f-29c1-4d9a-a9d0-3df21ec2f628 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 22:35:04.396460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:04.396460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:04.396460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:04.396460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:04.396460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:04.396460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:04.396460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:04.396460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:04.396460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:04.396460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:04.396460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:04.396460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:35:04.396460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:35:04.396460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:35:04.396460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:35:04.397661 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:04.397661 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:04.397661 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:35:04.397661 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:04.397661 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:35:04.397661 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:04.397661 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:04.397661 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:04.397661 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:04.397661 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:04.397661 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:04.397661 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:04.397661 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:04.397661 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:04.397661 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:04.397661 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:04.397661 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:04.397661 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:04.399006 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:04.399006 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:04.399006 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:35:04.399006 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:35:04.399006 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:35:04.399006 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:35:04.399006 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:35:04.399006 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:35:04.399006 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 22:35:04.399006 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:04.402255 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 285/576] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:35:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:04.403115 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5b12f-29c1-4d9a-a9d0-3df21ec2f628 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:04.416608 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b448b-6bcb-45f5-b8d8-a9b7a2ac5082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:04.417249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b448b-6bcb-45f5-b8d8-a9b7a2ac5082 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:04.417554 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b448b-6bcb-45f5-b8d8-a9b7a2ac5082 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:04.417830 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b448b-6bcb-45f5-b8d8-a9b7a2ac5082 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:04.422239 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:04.425549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b448b-6bcb-45f5-b8d8-a9b7a2ac5082 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:04.427440 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b448b-6bcb-45f5-b8d8-a9b7a2ac5082 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:04.439610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-347b448b-6bcb-45f5-b8d8-a9b7a2ac5082 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:04.439610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:04.439610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:04.439610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:04.439610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:04.439610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:04.439610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:04.439610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:04.439610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:04.439610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:04.439610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:04.439610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:04.439610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:35:04.439610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:35:04.439610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:35:04.440672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:35:04.440672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:04.440672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:04.440672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:04.440672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:04.440672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:04.440672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:04.440672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:04.440672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:04.440672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:04.440672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:04.440672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:04.440672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:04.440672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:04.440672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:04.440672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:04.440672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:04.442714 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:35:04.442714 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:35:04.442714 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:35:04.442714 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:35:04.442714 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:35:04.442714 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:35:04.442714 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:04.442714 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:04.442714 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 292/577] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:35:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:04.442714 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618806a1-0bb1-4671-b5f4-1aa7bf190210 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:04.443312 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618806a1-0bb1-4671-b5f4-1aa7bf190210 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:04.443312 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618806a1-0bb1-4671-b5f4-1aa7bf190210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:04.462516 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-618806a1-0bb1-4671-b5f4-1aa7bf190210 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:04.463697 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-618806a1-0bb1-4671-b5f4-1aa7bf190210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:04.528769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:04.535556 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5b12f-29c1-4d9a-a9d0-3df21ec2f628 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:04.549305 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 293/578] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:35:04.549305 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618806a1-0bb1-4671-b5f4-1aa7bf190210 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:04.583133 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22143101-e19a-4f6f-bcae-23f776c13630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:04.583835 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22143101-e19a-4f6f-bcae-23f776c13630 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:04.584195 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22143101-e19a-4f6f-bcae-23f776c13630 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:04.584542 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22143101-e19a-4f6f-bcae-23f776c13630 None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:04.585333 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22143101-e19a-4f6f-bcae-23f776c13630 None admin] RBAC: Adding query filter params (name=service) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:04.585710 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22143101-e19a-4f6f-bcae-23f776c13630 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:04.587092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22143101-e19a-4f6f-bcae-23f776c13630 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:04.596214 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 286/579] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 10 22:35:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:04.601957 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22143101-e19a-4f6f-bcae-23f776c13630 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:04.662326 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618806a1-0bb1-4671-b5f4-1aa7bf190210 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:04.686512 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:04.700355 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b9744-9d4a-4a59-befb-a56e5a7b258f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:04.700579 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b9744-9d4a-4a59-befb-a56e5a7b258f None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:04.700802 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b9744-9d4a-4a59-befb-a56e5a7b258f None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:04.701038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b9744-9d4a-4a59-befb-a56e5a7b258f None admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:04.701872 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b9744-9d4a-4a59-befb-a56e5a7b258f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:04.702103 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b9744-9d4a-4a59-befb-a56e5a7b258f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:04.703539 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b9744-9d4a-4a59-befb-a56e5a7b258f None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:04.714965 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22143101-e19a-4f6f-bcae-23f776c13630 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:04.726778 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 294/580] 10.222.0.76 () {56 vars in 1137 bytes} [Wed Aug 10 22:35:04 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:04.727925 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b9744-9d4a-4a59-befb-a56e5a7b258f None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:04.755109 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd00cbe-a045-4a64-bddc-10b5b8ef906b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:04.756076 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd00cbe-a045-4a64-bddc-10b5b8ef906b None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:04.756576 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd00cbe-a045-4a64-bddc-10b5b8ef906b None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:04.756968 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd00cbe-a045-4a64-bddc-10b5b8ef906b None admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:04.757807 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd00cbe-a045-4a64-bddc-10b5b8ef906b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:04.758178 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd00cbe-a045-4a64-bddc-10b5b8ef906b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:04.759673 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd00cbe-a045-4a64-bddc-10b5b8ef906b None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:04.772686 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 287/581] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:35:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:04.773584 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd00cbe-a045-4a64-bddc-10b5b8ef906b None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:04.786667 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:04.848019 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b9744-9d4a-4a59-befb-a56e5a7b258f None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:04.883383 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9745cb2-f61d-4595-b7dc-ae059a4d33f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:04.884020 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9745cb2-f61d-4595-b7dc-ae059a4d33f3 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:04.884302 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9745cb2-f61d-4595-b7dc-ae059a4d33f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:04.884572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9745cb2-f61d-4595-b7dc-ae059a4d33f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:04.887890 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd00cbe-a045-4a64-bddc-10b5b8ef906b None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:04.895795 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9745cb2-f61d-4595-b7dc-ae059a4d33f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:35:04.900396 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-d9745cb2-f61d-4595-b7dc-ae059a4d33f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:35:04.900396 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:04.900396 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:04.900396 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:04.900396 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:04.900396 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:04.900396 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:04.900396 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:04.900396 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:04.900396 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:04.900396 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:04.900396 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:04.900396 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:35:04.900396 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:35:04.900396 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:35:04.900396 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:35:04.901354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:35:04.901354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:35:04.901354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:35:04.901354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:35:04.901354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:04.901354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:04.901354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:04.901354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:04.901354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:04.901354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:04.901354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:04.901354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:04.901354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:04.901354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:04.901354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:04.901354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:04.902236 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:04.902236 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:04.902236 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:04.902236 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:04.902236 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:35:04.902236 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:35:04.902236 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:35:04.902236 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:35:04.902236 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:35:04.902236 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:35:04.902236 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:35:04.902236 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:04.905593 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 295/582] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 10 22:35:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:04.907203 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9745cb2-f61d-4595-b7dc-ae059a4d33f3 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:04.932433 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470b91be-0819-4ed8-a04a-958ec95ffb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:04.933072 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b91be-0819-4ed8-a04a-958ec95ffb2f None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:04.933387 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b91be-0819-4ed8-a04a-958ec95ffb2f None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:04.933660 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b91be-0819-4ed8-a04a-958ec95ffb2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:04.935586 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:04.945978 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470b91be-0819-4ed8-a04a-958ec95ffb2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:04.947437 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470b91be-0819-4ed8-a04a-958ec95ffb2f None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:04.963041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-470b91be-0819-4ed8-a04a-958ec95ffb2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:04.963041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:04.963041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:04.963041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:04.963041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:04.963041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:04.963041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:04.963041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:04.963041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:04.963041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:04.963041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:04.963041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:04.963041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:04.963041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:04.963041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:04.963041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:04.964142 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:04.964142 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:04.964142 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:04.964142 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:04.964142 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:04.964142 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:04.964142 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:04.964142 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:04.964142 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:04.964142 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:04.964142 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:04.964142 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:04.964142 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:04.964142 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:04.964142 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:04.964142 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:04.964142 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:04.965355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:04.965355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:04.965355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:04.965355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 288/583] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:04.965355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470b91be-0819-4ed8-a04a-958ec95ffb2f None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:05.025632 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9745cb2-f61d-4595-b7dc-ae059a4d33f3 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:05.037797 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:05.064090 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b0eea-66f0-4e6a-bda2-926a295f92f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:05.064896 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b0eea-66f0-4e6a-bda2-926a295f92f7 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:05.065704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b0eea-66f0-4e6a-bda2-926a295f92f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:05.066724 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b0eea-66f0-4e6a-bda2-926a295f92f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:05.077416 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b0eea-66f0-4e6a-bda2-926a295f92f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:05.078474 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470b91be-0819-4ed8-a04a-958ec95ffb2f None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:05.079066 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b0eea-66f0-4e6a-bda2-926a295f92f7 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:05.102012 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-ef5b0eea-66f0-4e6a-bda2-926a295f92f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:05.102012 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:05.102012 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:05.102012 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:05.102012 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:05.102012 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:05.102012 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:05.102012 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:05.102012 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:05.102012 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:05.102012 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:05.102012 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:05.102012 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:05.102012 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:05.102012 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:05.102012 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:05.103198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:05.103198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:05.103198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:05.103198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:05.103198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:05.103198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:05.103198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:05.103198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:05.103198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:05.103198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:05.103198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:05.103198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:05.103198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:05.103198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:05.103198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:05.103198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:05.103198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:05.104116 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:05.104116 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:05.104116 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:05.109661 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 296/584] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:05.110731 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b0eea-66f0-4e6a-bda2-926a295f92f7 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:05.121739 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a7ac15-65dc-4f55-8f0e-5131f0a6a24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:05.122485 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7ac15-65dc-4f55-8f0e-5131f0a6a24f None admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:05.122843 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7ac15-65dc-4f55-8f0e-5131f0a6a24f None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:05.123189 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7ac15-65dc-4f55-8f0e-5131f0a6a24f None admin] PATH_INFO: `/v3/projects/3caa2ea846f7449b8c483788fc5846c8/users/b22838a8ad3949e59cdbaaa1801cd07f/roles/5dfd0f952cd641bbb2257860c4c892ac` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:05.139068 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a7ac15-65dc-4f55-8f0e-5131f0a6a24f None admin] RBAC: Authorizing `identity:create_grant(project_id=3caa2ea846f7449b8c483788fc5846c8, user_id=b22838a8ad3949e59cdbaaa1801cd07f, role_id=5dfd0f952cd641bbb2257860c4c892ac)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:05.140609 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a7ac15-65dc-4f55-8f0e-5131f0a6a24f None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:05.156517 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 289/585] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 10 22:35:04 2022] PUT /identity/v3/projects/3caa2ea846f7449b8c483788fc5846c8/users/b22838a8ad3949e59cdbaaa1801cd07f/roles/5dfd0f952cd641bbb2257860c4c892ac => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:35:05.157386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a7ac15-65dc-4f55-8f0e-5131f0a6a24f None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:05.237691 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:05.239860 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b0eea-66f0-4e6a-bda2-926a295f92f7 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:05.285488 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861da7f0-7b09-4811-b3ca-0ebff029f6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:05.286643 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861da7f0-7b09-4811-b3ca-0ebff029f6e8 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:05.286920 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861da7f0-7b09-4811-b3ca-0ebff029f6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:05.287194 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861da7f0-7b09-4811-b3ca-0ebff029f6e8 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:05.287877 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861da7f0-7b09-4811-b3ca-0ebff029f6e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:05.288176 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861da7f0-7b09-4811-b3ca-0ebff029f6e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:05.289604 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861da7f0-7b09-4811-b3ca-0ebff029f6e8 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:05.302149 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 297/586] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 10 22:35:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:05.306285 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861da7f0-7b09-4811-b3ca-0ebff029f6e8 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:05.327781 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a7ac15-65dc-4f55-8f0e-5131f0a6a24f None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:05.339078 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:05.375321 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c02770-ba0a-41d4-a64d-bfc83ab59a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:05.376313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c02770-ba0a-41d4-a64d-bfc83ab59a8c None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:05.376940 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c02770-ba0a-41d4-a64d-bfc83ab59a8c None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:05.377398 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c02770-ba0a-41d4-a64d-bfc83ab59a8c None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:05.378765 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c02770-ba0a-41d4-a64d-bfc83ab59a8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:05.379879 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c02770-ba0a-41d4-a64d-bfc83ab59a8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:05.381745 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c02770-ba0a-41d4-a64d-bfc83ab59a8c None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:05.398314 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 290/587] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:05.399866 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c02770-ba0a-41d4-a64d-bfc83ab59a8c None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:05.432948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861da7f0-7b09-4811-b3ca-0ebff029f6e8 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:05.482106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21e9d4-7e78-42e9-a666-a3037735b1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:05.483007 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21e9d4-7e78-42e9-a666-a3037735b1a5 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:05.483529 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21e9d4-7e78-42e9-a666-a3037735b1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:05.484114 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21e9d4-7e78-42e9-a666-a3037735b1a5 None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:05.485544 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e21e9d4-7e78-42e9-a666-a3037735b1a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:05.486008 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e21e9d4-7e78-42e9-a666-a3037735b1a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:05.488635 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e21e9d4-7e78-42e9-a666-a3037735b1a5 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:05.506453 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 298/588] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:05.507671 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21e9d4-7e78-42e9-a666-a3037735b1a5 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:05.525678 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:05.533907 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c02770-ba0a-41d4-a64d-bfc83ab59a8c None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:05.573695 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b79ec4e-c248-4725-810f-e70eeae683b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:05.574756 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b79ec4e-c248-4725-810f-e70eeae683b4 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:05.575243 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b79ec4e-c248-4725-810f-e70eeae683b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:05.575616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b79ec4e-c248-4725-810f-e70eeae683b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:05.587826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b79ec4e-c248-4725-810f-e70eeae683b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:05.589502 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b79ec4e-c248-4725-810f-e70eeae683b4 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:05.606351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-9b79ec4e-c248-4725-810f-e70eeae683b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:05.606351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:05.606351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:05.606351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:05.606351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:05.606351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:05.606351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:05.606351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:05.606351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:05.606351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:05.606351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:05.606351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:05.606351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:05.606351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:05.606351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:05.606351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:05.607564 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:05.607564 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:05.607564 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:05.607564 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:05.607564 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:05.607564 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:05.607564 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:05.607564 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:05.607564 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:05.607564 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:05.607564 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:05.607564 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:05.607564 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:05.607564 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:05.607564 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:05.607564 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:05.607564 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:05.608660 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:05.608660 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:05.608660 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:05.610115 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 291/589] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:05.611002 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b79ec4e-c248-4725-810f-e70eeae683b4 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:05.621905 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:05.643845 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21e9d4-7e78-42e9-a666-a3037735b1a5 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:05.692363 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4d0edc-a475-4c8c-ac5f-1cc5beeb50a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:05.693309 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d0edc-a475-4c8c-ac5f-1cc5beeb50a4 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:05.693989 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d0edc-a475-4c8c-ac5f-1cc5beeb50a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:05.694450 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d0edc-a475-4c8c-ac5f-1cc5beeb50a4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:05.705007 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4d0edc-a475-4c8c-ac5f-1cc5beeb50a4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:05.707776 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4d0edc-a475-4c8c-ac5f-1cc5beeb50a4 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:05.725060 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-9d4d0edc-a475-4c8c-ac5f-1cc5beeb50a4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 22:35:05.725060 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:05.725060 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:05.725060 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:05.725060 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:05.725060 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:05.725060 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:05.725060 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:05.725060 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:05.725060 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:05.725060 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:05.725060 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:05.725060 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:35:05.725060 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:35:05.725060 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:35:05.725060 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:35:05.726346 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:05.726346 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:05.726346 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:35:05.726346 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:05.726346 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:35:05.726346 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:05.726346 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:05.726346 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:05.726346 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:05.726346 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:05.726346 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:05.726346 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:05.726346 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:05.726346 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:05.726346 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:05.726346 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:05.726346 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:05.726346 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:05.727510 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:05.727510 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:05.727510 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:35:05.727510 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:35:05.727510 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:35:05.727510 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:35:05.727510 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:35:05.727510 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:35:05.727510 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 22:35:05.727510 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:05.730548 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 299/590] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:35:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:05.731489 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4d0edc-a475-4c8c-ac5f-1cc5beeb50a4 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:05.736764 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b79ec4e-c248-4725-810f-e70eeae683b4 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:05.779128 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f65462-9b98-4a54-b81c-f5333d7bc00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:05.780599 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f65462-9b98-4a54-b81c-f5333d7bc00f None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:05.780599 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f65462-9b98-4a54-b81c-f5333d7bc00f None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:05.780599 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f65462-9b98-4a54-b81c-f5333d7bc00f None admin] PATH_INFO: `/v3/projects/service` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:05.788967 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f65462-9b98-4a54-b81c-f5333d7bc00f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:05.790662 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f65462-9b98-4a54-b81c-f5333d7bc00f None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:05.803137 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-97f65462-9b98-4a54-b81c-f5333d7bc00f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:05.803137 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:05.803137 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:05.803137 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:05.803137 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:05.803137 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:05.803137 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:05.803137 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:05.803137 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:05.803137 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:05.803137 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:05.803137 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:05.803137 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:35:05.803137 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:35:05.803137 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:35:05.804213 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:35:05.804213 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:05.804213 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:05.804213 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:05.804213 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:05.804213 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:05.804213 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:05.804213 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:05.804213 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:05.804213 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:05.804213 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:05.804213 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:05.804213 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:05.804213 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:05.804213 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:05.804213 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:05.804213 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:05.805267 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:35:05.805267 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:35:05.805267 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:35:05.805267 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:35:05.805267 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:35:05.805267 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:35:05.805267 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:05.805267 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:05.809627 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 292/591] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:35:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:05.810717 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f65462-9b98-4a54-b81c-f5333d7bc00f None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:05.827028 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:05.879703 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4d0edc-a475-4c8c-ac5f-1cc5beeb50a4 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:05.927360 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f65462-9b98-4a54-b81c-f5333d7bc00f None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:05.928443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ff806-9435-435e-804e-bcc16a983e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:05.929534 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ff806-9435-435e-804e-bcc16a983e97 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:05.930001 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ff806-9435-435e-804e-bcc16a983e97 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:05.931332 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ff806-9435-435e-804e-bcc16a983e97 None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:05.932156 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890ff806-9435-435e-804e-bcc16a983e97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:05.932610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890ff806-9435-435e-804e-bcc16a983e97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:05.934175 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890ff806-9435-435e-804e-bcc16a983e97 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:05.951957 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 300/592] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:05.953386 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ff806-9435-435e-804e-bcc16a983e97 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:05.969667 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:05.977192 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c5a5c-1fad-43dd-bc54-305b36b0795e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:05.982055 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c5a5c-1fad-43dd-bc54-305b36b0795e None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:05.982519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c5a5c-1fad-43dd-bc54-305b36b0795e None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:05.982960 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c5a5c-1fad-43dd-bc54-305b36b0795e None admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:05.983991 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5c5a5c-1fad-43dd-bc54-305b36b0795e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:05.984466 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5c5a5c-1fad-43dd-bc54-305b36b0795e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:05.986122 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5c5a5c-1fad-43dd-bc54-305b36b0795e None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:06.006552 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 293/593] 10.222.0.76 () {56 vars in 1137 bytes} [Wed Aug 10 22:35:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:06.007741 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c5a5c-1fad-43dd-bc54-305b36b0795e None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:06.090469 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:06.092749 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ff806-9435-435e-804e-bcc16a983e97 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:06.132478 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c32a54d-f631-4878-a775-e00cfbce9626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:06.133777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32a54d-f631-4878-a775-e00cfbce9626 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:06.134261 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32a54d-f631-4878-a775-e00cfbce9626 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:06.134837 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32a54d-f631-4878-a775-e00cfbce9626 None admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:06.136169 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c32a54d-f631-4878-a775-e00cfbce9626 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:06.136766 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c32a54d-f631-4878-a775-e00cfbce9626 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:06.138375 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c5a5c-1fad-43dd-bc54-305b36b0795e None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:06.138623 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c32a54d-f631-4878-a775-e00cfbce9626 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:06.154037 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 301/594] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:35:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:06.155487 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c32a54d-f631-4878-a775-e00cfbce9626 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:06.188648 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4880de9b-fa64-4442-b230-045e52c5f731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:06.189422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4880de9b-fa64-4442-b230-045e52c5f731 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:06.189684 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4880de9b-fa64-4442-b230-045e52c5f731 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:06.190057 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4880de9b-fa64-4442-b230-045e52c5f731 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:06.201883 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4880de9b-fa64-4442-b230-045e52c5f731 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:06.203507 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4880de9b-fa64-4442-b230-045e52c5f731 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:06.220436 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-4880de9b-fa64-4442-b230-045e52c5f731 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 22:35:06.220436 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:06.220436 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:06.220436 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:06.220436 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:06.220436 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:06.220436 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:06.220436 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:06.220436 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:06.220436 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:06.220436 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:06.220436 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:06.220436 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:35:06.220436 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:35:06.220436 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:35:06.220436 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:35:06.221659 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:06.221659 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:06.221659 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:35:06.221659 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:06.221659 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:35:06.221659 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:06.221659 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:06.221659 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:06.221659 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:06.221659 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:06.221659 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:06.221659 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:06.221659 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:06.221659 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:06.221659 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:06.221659 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:06.221659 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:06.221659 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:06.222623 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:06.222623 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:06.222623 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:35:06.222623 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:35:06.222623 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:35:06.222623 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:35:06.222623 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:35:06.222623 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:35:06.222623 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 22:35:06.222623 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:06.222623 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 294/595] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 10 22:35:06 2022] GET /identity/v3/users/neutron => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:06.223447 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4880de9b-fa64-4442-b230-045e52c5f731 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:06.236345 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:06.287664 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c32a54d-f631-4878-a775-e00cfbce9626 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:06.331853 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e8ea78-40a9-4161-8548-00b4506fcc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:06.333143 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8ea78-40a9-4161-8548-00b4506fcc6d None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:06.333661 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8ea78-40a9-4161-8548-00b4506fcc6d None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:06.334118 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8ea78-40a9-4161-8548-00b4506fcc6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:06.349068 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e8ea78-40a9-4161-8548-00b4506fcc6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:06.351033 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e8ea78-40a9-4161-8548-00b4506fcc6d None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:06.360665 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4880de9b-fa64-4442-b230-045e52c5f731 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:06.367674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-67e8ea78-40a9-4161-8548-00b4506fcc6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:06.367674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:06.367674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:06.367674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:06.367674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:06.367674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:06.367674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:06.367674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:06.367674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:06.367674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:06.367674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:06.367674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:06.367674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:06.367674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:06.367674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:06.367674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:06.368606 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:06.368606 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:06.368606 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:06.368606 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:06.368606 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:06.368606 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:06.368606 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:06.368606 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:06.368606 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:06.368606 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:06.368606 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:06.368606 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:06.368606 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:06.368606 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:06.368606 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:06.368606 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:06.368606 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:06.369525 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:06.369525 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:06.369525 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:06.374995 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 302/596] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:06.374995 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e8ea78-40a9-4161-8548-00b4506fcc6d None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:06.389409 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:06.400704 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffda33e6-4bbd-4413-875d-1fbf497fc03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:06.401476 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda33e6-4bbd-4413-875d-1fbf497fc03b None admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:06.401759 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda33e6-4bbd-4413-875d-1fbf497fc03b None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:06.402085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda33e6-4bbd-4413-875d-1fbf497fc03b None admin] PATH_INFO: `/v3/projects/3caa2ea846f7449b8c483788fc5846c8/users/a7f0dd6c9e2d4315966e90cd3b927b13/roles/5dfd0f952cd641bbb2257860c4c892ac` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:06.421868 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffda33e6-4bbd-4413-875d-1fbf497fc03b None admin] RBAC: Authorizing `identity:create_grant(project_id=3caa2ea846f7449b8c483788fc5846c8, user_id=a7f0dd6c9e2d4315966e90cd3b927b13, role_id=5dfd0f952cd641bbb2257860c4c892ac)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:06.423328 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffda33e6-4bbd-4413-875d-1fbf497fc03b None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:06.440759 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 295/597] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 10 22:35:06 2022] PUT /identity/v3/projects/3caa2ea846f7449b8c483788fc5846c8/users/a7f0dd6c9e2d4315966e90cd3b927b13/roles/5dfd0f952cd641bbb2257860c4c892ac => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:35:06.443250 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6409486f-8e99-4bc2-b959-8ebaea63c46d None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:06.443705 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6409486f-8e99-4bc2-b959-8ebaea63c46d None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:06.444129 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6409486f-8e99-4bc2-b959-8ebaea63c46d None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:06.446102 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 296/598] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:06.447088 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6409486f-8e99-4bc2-b959-8ebaea63c46d None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:06.515818 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e8ea78-40a9-4161-8548-00b4506fcc6d None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:06.537715 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:06.563527 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b31ed7-74ab-47c8-8cae-cb2442718051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:06.565131 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b31ed7-74ab-47c8-8cae-cb2442718051 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:06.565456 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b31ed7-74ab-47c8-8cae-cb2442718051 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:06.565731 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b31ed7-74ab-47c8-8cae-cb2442718051 None admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:06.567016 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b31ed7-74ab-47c8-8cae-cb2442718051 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:06.567326 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b31ed7-74ab-47c8-8cae-cb2442718051 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:06.568781 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b31ed7-74ab-47c8-8cae-cb2442718051 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:06.595780 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 303/599] 10.222.0.76 () {56 vars in 1139 bytes} [Wed Aug 10 22:35:06 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:06.598072 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e7bfa-5242-4132-b27f-5c25ce15d338 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:06.598367 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e7bfa-5242-4132-b27f-5c25ce15d338 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:06.598695 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e7bfa-5242-4132-b27f-5c25ce15d338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:06.606674 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6409486f-8e99-4bc2-b959-8ebaea63c46d None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:06.624487 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-692e7bfa-5242-4132-b27f-5c25ce15d338 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:06.624487 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-692e7bfa-5242-4132-b27f-5c25ce15d338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:06.649492 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b595e3-fa9c-4c1c-a3c4-45db015615a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:06.650277 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b595e3-fa9c-4c1c-a3c4-45db015615a9 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:06.650398 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b595e3-fa9c-4c1c-a3c4-45db015615a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:06.650648 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b595e3-fa9c-4c1c-a3c4-45db015615a9 None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:06.651282 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b595e3-fa9c-4c1c-a3c4-45db015615a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:06.652226 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b595e3-fa9c-4c1c-a3c4-45db015615a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:06.652959 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b595e3-fa9c-4c1c-a3c4-45db015615a9 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:06.667255 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 297/600] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:06.668361 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b595e3-fa9c-4c1c-a3c4-45db015615a9 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:06.709926 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:06.730511 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 304/601] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:35:06.731587 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e7bfa-5242-4132-b27f-5c25ce15d338 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:06.807072 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b595e3-fa9c-4c1c-a3c4-45db015615a9 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:06.855407 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecdd331-7395-4193-b8a1-74823692316c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:06.857224 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecdd331-7395-4193-b8a1-74823692316c None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:06.857760 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecdd331-7395-4193-b8a1-74823692316c None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:06.858247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecdd331-7395-4193-b8a1-74823692316c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:06.873331 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecdd331-7395-4193-b8a1-74823692316c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:06.875399 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecdd331-7395-4193-b8a1-74823692316c None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:06.877581 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e7bfa-5242-4132-b27f-5c25ce15d338 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:06.883136 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:06.898754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-aecdd331-7395-4193-b8a1-74823692316c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:06.898754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:06.898754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:06.898754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:06.898754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:06.898754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:06.898754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:06.898754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:06.898754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:06.898754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:06.898754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:06.898754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:06.898754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:06.898754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:06.898754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:06.898754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:06.899747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:06.899747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:06.899747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:06.899747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:06.899747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:06.899747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:06.899747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:06.899747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:06.899747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:06.899747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:06.899747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:06.899747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:06.899747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:06.899747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:06.899747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:06.899747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:06.899747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:06.900619 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:06.900619 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:06.900619 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:06.909321 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 298/602] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:06.909321 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecdd331-7395-4193-b8a1-74823692316c None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:06.939925 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28358e9-5fcf-4e67-b6f4-42e07148d0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:06.941556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28358e9-5fcf-4e67-b6f4-42e07148d0ee None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:06.942136 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28358e9-5fcf-4e67-b6f4-42e07148d0ee None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:06.942608 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28358e9-5fcf-4e67-b6f4-42e07148d0ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:06.952267 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28358e9-5fcf-4e67-b6f4-42e07148d0ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:06.954668 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28358e9-5fcf-4e67-b6f4-42e07148d0ee None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:06.978713 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-e28358e9-5fcf-4e67-b6f4-42e07148d0ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:06.978713 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:06.978713 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:06.978713 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:06.978713 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:06.978713 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:06.978713 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:06.978713 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:06.978713 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:06.978713 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:06.978713 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:06.978713 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:06.978713 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:35:06.978713 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:35:06.978713 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:35:06.979718 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:35:06.979718 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:06.979718 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:06.979718 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:06.979718 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:06.979718 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:06.979718 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:06.979718 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:06.979718 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:06.979718 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:06.979718 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:06.979718 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:06.979718 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:06.979718 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:06.979718 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:06.979718 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:06.979718 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:06.980665 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:35:06.980665 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:35:06.980665 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:35:06.980665 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:35:06.980665 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:35:06.980665 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:35:06.980665 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:06.980665 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:06.983972 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 305/603] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:35:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:06.985038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28358e9-5fcf-4e67-b6f4-42e07148d0ee None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:07.049663 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:07.061012 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecdd331-7395-4193-b8a1-74823692316c None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:07.108443 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a9ce96-c635-4b94-b2c1-59a269cb2433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:07.109307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9ce96-c635-4b94-b2c1-59a269cb2433 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:07.109735 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9ce96-c635-4b94-b2c1-59a269cb2433 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:07.110116 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9ce96-c635-4b94-b2c1-59a269cb2433 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:07.121269 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a9ce96-c635-4b94-b2c1-59a269cb2433 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:35:07.124717 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28358e9-5fcf-4e67-b6f4-42e07148d0ee None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:07.126784 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-51a9ce96-c635-4b94-b2c1-59a269cb2433 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:35:07.126784 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:07.126784 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:07.126784 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:07.126784 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:07.126784 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:07.126784 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:07.126784 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:07.126784 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:07.126784 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:07.126784 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:07.126784 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:07.126784 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:35:07.126784 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:35:07.126784 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:35:07.126784 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:35:07.127857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:35:07.127857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:35:07.127857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:35:07.127857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:35:07.127857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:07.127857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:07.127857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:07.127857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:07.127857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:07.127857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:07.127857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:07.127857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:07.127857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:07.127857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:07.127857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:07.127857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:07.128900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:07.128900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:07.128900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:07.128900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:07.128900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:35:07.128900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:35:07.128900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:35:07.128900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:35:07.128900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:35:07.128900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:35:07.128900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:35:07.128900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:07.131061 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 299/604] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 10 22:35:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:07.132027 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a9ce96-c635-4b94-b2c1-59a269cb2433 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:07.167556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3ca9f0-cdac-4d57-9fe3-aa4e987006db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:07.168446 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ca9f0-cdac-4d57-9fe3-aa4e987006db None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:07.168869 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ca9f0-cdac-4d57-9fe3-aa4e987006db None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:07.169302 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ca9f0-cdac-4d57-9fe3-aa4e987006db None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:07.170237 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3ca9f0-cdac-4d57-9fe3-aa4e987006db None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:07.170783 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3ca9f0-cdac-4d57-9fe3-aa4e987006db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:07.172341 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3ca9f0-cdac-4d57-9fe3-aa4e987006db None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:07.190417 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 306/605] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:07.191640 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3ca9f0-cdac-4d57-9fe3-aa4e987006db None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:07.221203 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:07.243714 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a9ce96-c635-4b94-b2c1-59a269cb2433 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:07.281780 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e093a-c80d-43bc-a6d7-bf5b091ab23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:07.282685 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e093a-c80d-43bc-a6d7-bf5b091ab23a None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:07.283072 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e093a-c80d-43bc-a6d7-bf5b091ab23a None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:07.283450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e093a-c80d-43bc-a6d7-bf5b091ab23a None admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:07.284374 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108e093a-c80d-43bc-a6d7-bf5b091ab23a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:07.285454 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108e093a-c80d-43bc-a6d7-bf5b091ab23a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:07.287266 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108e093a-c80d-43bc-a6d7-bf5b091ab23a None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:07.301957 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 300/606] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:35:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:07.303117 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e093a-c80d-43bc-a6d7-bf5b091ab23a None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:07.308874 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3ca9f0-cdac-4d57-9fe3-aa4e987006db None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:07.314755 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:07.349446 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd9adb-0ab4-47ed-bee5-8b0f2a64808e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:07.350320 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd9adb-0ab4-47ed-bee5-8b0f2a64808e None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:07.350823 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd9adb-0ab4-47ed-bee5-8b0f2a64808e None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:07.351337 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd9adb-0ab4-47ed-bee5-8b0f2a64808e None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:07.352259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd9adb-0ab4-47ed-bee5-8b0f2a64808e None admin] RBAC: Adding query filter params (name=service) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:07.352871 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd9adb-0ab4-47ed-bee5-8b0f2a64808e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:07.354563 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd9adb-0ab4-47ed-bee5-8b0f2a64808e None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:07.363767 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 307/607] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 10 22:35:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:07.365196 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd9adb-0ab4-47ed-bee5-8b0f2a64808e None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:07.448407 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e093a-c80d-43bc-a6d7-bf5b091ab23a None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:07.460910 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:07.481223 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd9adb-0ab4-47ed-bee5-8b0f2a64808e None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:07.495738 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb87928-c82f-4c25-97c3-2eff74e5cf04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:07.496297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb87928-c82f-4c25-97c3-2eff74e5cf04 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:07.496574 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb87928-c82f-4c25-97c3-2eff74e5cf04 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:07.496840 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb87928-c82f-4c25-97c3-2eff74e5cf04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:07.504191 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb87928-c82f-4c25-97c3-2eff74e5cf04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:07.505788 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb87928-c82f-4c25-97c3-2eff74e5cf04 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:07.517320 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-8eb87928-c82f-4c25-97c3-2eff74e5cf04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:07.517320 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:07.517320 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:07.517320 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:07.517320 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:07.517320 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:07.517320 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:07.517320 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:07.517320 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:07.517320 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:07.517320 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:07.517320 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:07.517320 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:35:07.517320 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:35:07.517320 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:35:07.518230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:35:07.518230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:07.518230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:07.518230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:07.518230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:07.518230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:07.518230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:07.518230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:07.518230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:07.518230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:07.518230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:07.518230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:07.518230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:07.518230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:07.518230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:07.518230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:07.518230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:07.519820 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:35:07.519820 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:35:07.519820 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:35:07.519820 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:35:07.519820 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:35:07.519820 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:35:07.519820 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:07.519820 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:07.519820 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 301/608] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:35:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:07.519820 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb87928-c82f-4c25-97c3-2eff74e5cf04 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:07.525328 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c1697-5c48-498d-8868-3caa0e23cdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:07.526003 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c1697-5c48-498d-8868-3caa0e23cdc5 None admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:07.526283 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c1697-5c48-498d-8868-3caa0e23cdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:07.526554 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c1697-5c48-498d-8868-3caa0e23cdc5 None admin] PATH_INFO: `/v3/projects/3caa2ea846f7449b8c483788fc5846c8/users/dc7524edf5d5422f91d92fd91be5ee1e/roles/5dfd0f952cd641bbb2257860c4c892ac` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:07.543600 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9c1697-5c48-498d-8868-3caa0e23cdc5 None admin] RBAC: Authorizing `identity:create_grant(project_id=3caa2ea846f7449b8c483788fc5846c8, user_id=dc7524edf5d5422f91d92fd91be5ee1e, role_id=5dfd0f952cd641bbb2257860c4c892ac)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:07.545047 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9c1697-5c48-498d-8868-3caa0e23cdc5 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:07.565393 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 308/609] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 10 22:35:07 2022] PUT /identity/v3/projects/3caa2ea846f7449b8c483788fc5846c8/users/dc7524edf5d5422f91d92fd91be5ee1e/roles/5dfd0f952cd641bbb2257860c4c892ac => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:35:07.569219 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8c7b0-7fb9-4283-92dd-33137eaea74d None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:07.569846 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8c7b0-7fb9-4283-92dd-33137eaea74d None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:07.570326 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8c7b0-7fb9-4283-92dd-33137eaea74d None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:07.573304 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 309/610] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:07.574865 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8c7b0-7fb9-4283-92dd-33137eaea74d None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:07.607753 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:07.660071 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb87928-c82f-4c25-97c3-2eff74e5cf04 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:07.711175 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da430d8b-7940-4692-9ca6-d089d10b292f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:07.711876 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da430d8b-7940-4692-9ca6-d089d10b292f None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:07.712390 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da430d8b-7940-4692-9ca6-d089d10b292f None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:07.712714 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da430d8b-7940-4692-9ca6-d089d10b292f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:07.728992 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da430d8b-7940-4692-9ca6-d089d10b292f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:07.730561 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da430d8b-7940-4692-9ca6-d089d10b292f None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:07.738745 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8c7b0-7fb9-4283-92dd-33137eaea74d None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:07.750346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-da430d8b-7940-4692-9ca6-d089d10b292f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:07.750346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:07.750346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:07.750346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:07.750346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:07.750346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:07.750346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:07.750346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:07.750346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:07.750346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:07.750346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:07.750346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:07.750346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:07.750346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:07.750346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:07.750346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:07.751302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:07.751302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:07.751302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:07.751302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:07.751302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:07.751302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:07.751302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:07.751302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:07.751302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:07.751302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:07.751302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:07.751302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:07.751302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:07.751302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:07.751302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:07.751302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:07.751302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:07.752203 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:07.752203 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:07.752203 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:07.752203 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 302/611] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:07.752928 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ecedcb-e157-4833-95c9-8e75c55c2052 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:07.753209 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ecedcb-e157-4833-95c9-8e75c55c2052 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:07.758216 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ecedcb-e157-4833-95c9-8e75c55c2052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:07.788464 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-96ecedcb-e157-4833-95c9-8e75c55c2052 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:07.789795 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-96ecedcb-e157-4833-95c9-8e75c55c2052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:07.803819 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc408f-48cc-46a4-b895-fb7e67790718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:07.805355 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc408f-48cc-46a4-b895-fb7e67790718 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:07.805965 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc408f-48cc-46a4-b895-fb7e67790718 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:07.806443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc408f-48cc-46a4-b895-fb7e67790718 None admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:07.807608 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dc408f-48cc-46a4-b895-fb7e67790718 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:07.808161 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dc408f-48cc-46a4-b895-fb7e67790718 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:07.810564 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dc408f-48cc-46a4-b895-fb7e67790718 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:07.814325 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:07.829466 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 310/612] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:35:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:07.835612 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc408f-48cc-46a4-b895-fb7e67790718 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:07.877128 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 303/613] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:35:07.878423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ecedcb-e157-4833-95c9-8e75c55c2052 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:07.933706 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:07.969114 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc408f-48cc-46a4-b895-fb7e67790718 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:07.992710 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ecedcb-e157-4833-95c9-8e75c55c2052 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:08.006223 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf0cd26-c219-4608-a1d3-a305572d4e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:08.006523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0cd26-c219-4608-a1d3-a305572d4e9d None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:08.006771 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0cd26-c219-4608-a1d3-a305572d4e9d None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:08.007041 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0cd26-c219-4608-a1d3-a305572d4e9d None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:08.007826 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf0cd26-c219-4608-a1d3-a305572d4e9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:08.008155 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf0cd26-c219-4608-a1d3-a305572d4e9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:08.009533 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf0cd26-c219-4608-a1d3-a305572d4e9d None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:08.030461 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 311/614] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:08.035637 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf0cd26-c219-4608-a1d3-a305572d4e9d None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:08.039667 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c969c45c-a227-46ea-a470-cec43d1c6928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:08.040786 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969c45c-a227-46ea-a470-cec43d1c6928 None admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:08.041378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969c45c-a227-46ea-a470-cec43d1c6928 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:08.041877 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969c45c-a227-46ea-a470-cec43d1c6928 None admin] PATH_INFO: `/v3/projects/3caa2ea846f7449b8c483788fc5846c8/users/7fcb5a674ff94b3d8f79efbbe4959069/roles/5dfd0f952cd641bbb2257860c4c892ac` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:08.060836 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c969c45c-a227-46ea-a470-cec43d1c6928 None admin] RBAC: Authorizing `identity:create_grant(project_id=3caa2ea846f7449b8c483788fc5846c8, user_id=7fcb5a674ff94b3d8f79efbbe4959069, role_id=5dfd0f952cd641bbb2257860c4c892ac)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:08.062858 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c969c45c-a227-46ea-a470-cec43d1c6928 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:08.081377 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 304/615] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 10 22:35:07 2022] PUT /identity/v3/projects/3caa2ea846f7449b8c483788fc5846c8/users/7fcb5a674ff94b3d8f79efbbe4959069/roles/5dfd0f952cd641bbb2257860c4c892ac => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:35:08.085362 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c969c45c-a227-46ea-a470-cec43d1c6928 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:08.138897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:08.211101 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf0cd26-c219-4608-a1d3-a305572d4e9d None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:08.223476 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:08.227347 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c969c45c-a227-46ea-a470-cec43d1c6928 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:08.277876 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aced7091-f5a2-4b20-81c2-4d27350636b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:08.278448 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced7091-f5a2-4b20-81c2-4d27350636b2 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:08.278725 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced7091-f5a2-4b20-81c2-4d27350636b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:08.278994 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced7091-f5a2-4b20-81c2-4d27350636b2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:08.283550 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c28e0-a8df-4369-8e10-04bbfbd519dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:08.284460 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c28e0-a8df-4369-8e10-04bbfbd519dc None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:08.285533 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c28e0-a8df-4369-8e10-04bbfbd519dc None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:08.286089 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c28e0-a8df-4369-8e10-04bbfbd519dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:08.288098 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aced7091-f5a2-4b20-81c2-4d27350636b2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:08.289586 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aced7091-f5a2-4b20-81c2-4d27350636b2 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:08.300207 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5c28e0-a8df-4369-8e10-04bbfbd519dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:35:08.304807 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-aced7091-f5a2-4b20-81c2-4d27350636b2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 22:35:08.304807 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:08.304807 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:08.304807 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:08.304807 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:08.304807 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:08.304807 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:08.304807 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:08.304807 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:08.304807 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:08.304807 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:08.304807 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:08.304807 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:35:08.304807 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:35:08.304807 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:35:08.304807 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:35:08.305925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:08.305925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:08.305925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:35:08.305925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:08.305925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:35:08.305925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:08.305925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:08.305925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:08.305925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:08.305925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:08.305925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:08.305925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:08.305925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:08.305925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:08.305925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:08.305925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:08.305925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:08.305925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:08.306864 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:08.306864 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:08.306864 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:35:08.306864 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:35:08.306864 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:35:08.306864 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:35:08.306864 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:35:08.306864 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:35:08.306864 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 22:35:08.306864 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:08.310085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 305/616] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:35:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:08.313587 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aced7091-f5a2-4b20-81c2-4d27350636b2 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:08.313730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-cf5c28e0-a8df-4369-8e10-04bbfbd519dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:35:08.313730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:08.313730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:08.313730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:08.313730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:08.313730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:08.313730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:08.313730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:08.313730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:08.313730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:08.313730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:08.313730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:08.313730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:35:08.313730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:35:08.313730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:35:08.313730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:35:08.314792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:35:08.314792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:35:08.314792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:35:08.314792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:35:08.314792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:08.314792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:08.314792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:08.314792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:08.314792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:08.314792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:08.314792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:08.314792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:08.314792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:08.314792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:08.314792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:08.314792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:08.316546 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:08.316546 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:08.316546 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:08.316546 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:08.316546 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:35:08.316546 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:35:08.316546 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:35:08.316546 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:35:08.316546 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:35:08.316546 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:35:08.316546 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:35:08.316546 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:08.318555 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 312/617] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 10 22:35:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:08.324257 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c28e0-a8df-4369-8e10-04bbfbd519dc None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:08.446668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aced7091-f5a2-4b20-81c2-4d27350636b2 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:08.467047 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c28e0-a8df-4369-8e10-04bbfbd519dc None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:08.484968 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:08.498044 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e788c3e-68e7-4ec3-bb73-7ed46182756f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:08.498884 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e788c3e-68e7-4ec3-bb73-7ed46182756f None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:08.500262 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e788c3e-68e7-4ec3-bb73-7ed46182756f None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:08.500700 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e788c3e-68e7-4ec3-bb73-7ed46182756f None admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:08.501805 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e788c3e-68e7-4ec3-bb73-7ed46182756f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:08.503310 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e788c3e-68e7-4ec3-bb73-7ed46182756f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:08.505697 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e788c3e-68e7-4ec3-bb73-7ed46182756f None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:08.524957 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 306/618] 10.222.0.76 () {56 vars in 1135 bytes} [Wed Aug 10 22:35:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:35:08.531844 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e788c3e-68e7-4ec3-bb73-7ed46182756f None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:08.532915 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada223e-b5eb-4210-8de9-553c6b8e45d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:08.533558 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada223e-b5eb-4210-8de9-553c6b8e45d4 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:08.534345 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada223e-b5eb-4210-8de9-553c6b8e45d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:08.534629 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada223e-b5eb-4210-8de9-553c6b8e45d4 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:08.535335 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ada223e-b5eb-4210-8de9-553c6b8e45d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:08.535634 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ada223e-b5eb-4210-8de9-553c6b8e45d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:08.537221 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ada223e-b5eb-4210-8de9-553c6b8e45d4 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:08.545929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 313/619] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 10 22:35:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:08.553305 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada223e-b5eb-4210-8de9-553c6b8e45d4 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:08.569843 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:08.666439 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada223e-b5eb-4210-8de9-553c6b8e45d4 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:08.703571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c239a-e359-442d-8cb3-2e99cde2f2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:08.704372 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c239a-e359-442d-8cb3-2e99cde2f2c8 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:08.704772 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c239a-e359-442d-8cb3-2e99cde2f2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:08.705151 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c239a-e359-442d-8cb3-2e99cde2f2c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:08.717077 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860c239a-e359-442d-8cb3-2e99cde2f2c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:08.718848 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860c239a-e359-442d-8cb3-2e99cde2f2c8 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:08.734984 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-860c239a-e359-442d-8cb3-2e99cde2f2c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:08.734984 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:08.734984 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:08.734984 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:08.734984 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:08.734984 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:08.734984 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:08.734984 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:08.734984 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:08.734984 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:08.734984 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:08.734984 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:08.734984 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:08.734984 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:08.734984 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:08.734984 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:08.736214 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:08.736214 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:08.736214 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:08.736214 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:08.736214 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:08.736214 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:08.736214 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:08.736214 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:08.736214 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:08.736214 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:08.736214 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:08.736214 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:08.736214 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:08.736214 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:08.736214 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:08.736214 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:08.736214 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:08.737074 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:08.737074 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:08.737074 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:08.741841 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 314/620] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:35:08.750404 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c239a-e359-442d-8cb3-2e99cde2f2c8 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:08.765355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e788c3e-68e7-4ec3-bb73-7ed46182756f None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:08.809176 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:08.831709 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f808c1d3-57d0-4a8a-b41a-be898ab685d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:08.832546 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808c1d3-57d0-4a8a-b41a-be898ab685d6 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:08.832823 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808c1d3-57d0-4a8a-b41a-be898ab685d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:08.833103 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808c1d3-57d0-4a8a-b41a-be898ab685d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:08.855610 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f808c1d3-57d0-4a8a-b41a-be898ab685d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:08.857084 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f808c1d3-57d0-4a8a-b41a-be898ab685d6 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:08.879299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-f808c1d3-57d0-4a8a-b41a-be898ab685d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:08.879299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:08.879299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:08.879299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:08.879299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:08.879299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:08.879299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:08.879299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:08.879299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:08.879299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:08.879299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:08.879299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:08.879299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:08.879299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:08.879299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:08.879299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:08.880345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:08.880345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:08.880345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:08.880345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:08.880345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:08.880345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:08.880345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:08.880345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:08.880345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:08.880345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:08.880345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:08.880345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:08.880345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:08.880345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:08.880345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:08.880345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:08.880345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:08.881228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:08.881228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:08.881228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:08.881228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 307/621] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:08.882834 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7c49c-0204-4719-b393-d65c0616b83d None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:08.882834 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7c49c-0204-4719-b393-d65c0616b83d None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:08.882834 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7c49c-0204-4719-b393-d65c0616b83d None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:08.888049 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 308/622] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:08.888884 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7c49c-0204-4719-b393-d65c0616b83d None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:08.913959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c239a-e359-442d-8cb3-2e99cde2f2c8 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:08.949240 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df7d13-007d-4d91-8c32-72b8f968e5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:08.950141 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df7d13-007d-4d91-8c32-72b8f968e5a0 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:08.950417 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df7d13-007d-4d91-8c32-72b8f968e5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:08.950685 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df7d13-007d-4d91-8c32-72b8f968e5a0 None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:08.951358 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1df7d13-007d-4d91-8c32-72b8f968e5a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:08.951659 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1df7d13-007d-4d91-8c32-72b8f968e5a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:08.953072 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1df7d13-007d-4d91-8c32-72b8f968e5a0 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:08.965843 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 315/623] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:08.966299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:08.971873 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc63cc0-693e-4d7a-ba06-32f47df94747 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:08.972907 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc63cc0-693e-4d7a-ba06-32f47df94747 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:08.973344 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc63cc0-693e-4d7a-ba06-32f47df94747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:08.994238 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-abc63cc0-693e-4d7a-ba06-32f47df94747 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:08.995645 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-abc63cc0-693e-4d7a-ba06-32f47df94747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:09.002263 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7c49c-0204-4719-b393-d65c0616b83d None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:09.040511 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d448c5-0b33-4e08-9a15-ab63f2188693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:09.041127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d448c5-0b33-4e08-9a15-ab63f2188693 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:09.041538 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d448c5-0b33-4e08-9a15-ab63f2188693 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:09.041815 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d448c5-0b33-4e08-9a15-ab63f2188693 None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:09.042458 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d448c5-0b33-4e08-9a15-ab63f2188693 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:09.042759 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d448c5-0b33-4e08-9a15-ab63f2188693 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:09.044251 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d448c5-0b33-4e08-9a15-ab63f2188693 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:09.057111 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 309/624] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:09.058097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d448c5-0b33-4e08-9a15-ab63f2188693 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:09.071042 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 316/625] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:35:09.071862 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc63cc0-693e-4d7a-ba06-32f47df94747 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:09.095978 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:09.182368 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc63cc0-693e-4d7a-ba06-32f47df94747 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:09.196181 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d448c5-0b33-4e08-9a15-ab63f2188693 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:09.220244 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5155ac75-94b4-4489-b50c-d63b48387893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:09.221124 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5155ac75-94b4-4489-b50c-d63b48387893 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:09.221549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5155ac75-94b4-4489-b50c-d63b48387893 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:09.221948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5155ac75-94b4-4489-b50c-d63b48387893 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:09.229499 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:09.233855 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5155ac75-94b4-4489-b50c-d63b48387893 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:09.235562 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5155ac75-94b4-4489-b50c-d63b48387893 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:09.236592 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad437d2-6645-42e6-82bb-7bc05fffa4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:09.237243 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad437d2-6645-42e6-82bb-7bc05fffa4da None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:09.237558 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad437d2-6645-42e6-82bb-7bc05fffa4da None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:09.238510 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad437d2-6645-42e6-82bb-7bc05fffa4da None admin] PATH_INFO: `/v3/users/nova` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:09.247322 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad437d2-6645-42e6-82bb-7bc05fffa4da None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:09.248792 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad437d2-6645-42e6-82bb-7bc05fffa4da None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:09.249064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-5155ac75-94b4-4489-b50c-d63b48387893 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:09.249064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:09.249064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:09.249064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:09.249064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:09.249064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:09.249064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:09.249064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:09.249064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:09.249064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:09.249064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:09.249064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:09.249064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:35:09.249064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:35:09.249064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:35:09.251251 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:35:09.251251 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:09.251251 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:09.251251 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:09.251251 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:09.251251 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:09.251251 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:09.251251 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:09.251251 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:09.251251 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:09.251251 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:09.251251 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:09.251251 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:09.251251 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:09.251251 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:09.251251 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:09.251251 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:09.253620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:35:09.253620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:35:09.253620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:35:09.253620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:35:09.253620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:35:09.253620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:35:09.253620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:09.253620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:09.253620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 317/626] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:35:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:09.253620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5155ac75-94b4-4489-b50c-d63b48387893 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:09.261826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-bad437d2-6645-42e6-82bb-7bc05fffa4da None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:35:09.261826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:09.261826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:09.261826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:09.261826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:09.261826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:09.261826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:09.261826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:09.261826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:09.261826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:09.261826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:09.261826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:09.261826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:35:09.261826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:35:09.261826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:35:09.261826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:35:09.264085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:09.264085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:09.264085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:35:09.264085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:09.264085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:35:09.264085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:09.264085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:09.264085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:09.264085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:09.264085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:09.264085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:09.264085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:09.264085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:09.264085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:09.264085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:09.264085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:09.264085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:09.264085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:09.266616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:09.266616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:09.266616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:35:09.266616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:35:09.266616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:35:09.266616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:35:09.266616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:35:09.266616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:35:09.266616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:35:09.266616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:09.266616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 310/627] 10.222.0.76 () {56 vars in 1093 bytes} [Wed Aug 10 22:35:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:09.266616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad437d2-6645-42e6-82bb-7bc05fffa4da None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:09.345317 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:09.367242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5155ac75-94b4-4489-b50c-d63b48387893 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:09.375502 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad437d2-6645-42e6-82bb-7bc05fffa4da None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:09.404084 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc5481-ff36-4142-940b-0fa80ba2989d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:09.404916 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc5481-ff36-4142-940b-0fa80ba2989d None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:09.405089 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc5481-ff36-4142-940b-0fa80ba2989d None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:09.405385 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc5481-ff36-4142-940b-0fa80ba2989d None admin] PATH_INFO: `/v3/roles/service` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:09.413654 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c00ac-a0d8-46d7-b719-c24d652e529e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:09.414512 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c00ac-a0d8-46d7-b719-c24d652e529e None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:09.414948 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c00ac-a0d8-46d7-b719-c24d652e529e None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:09.415374 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c00ac-a0d8-46d7-b719-c24d652e529e None admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:09.416218 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11c00ac-a0d8-46d7-b719-c24d652e529e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:09.416419 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fc5481-ff36-4142-940b-0fa80ba2989d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:35:09.416842 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11c00ac-a0d8-46d7-b719-c24d652e529e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:09.418465 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11c00ac-a0d8-46d7-b719-c24d652e529e None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:09.421523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-e4fc5481-ff36-4142-940b-0fa80ba2989d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:35:09.421523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:09.421523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:09.421523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:09.421523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:09.421523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:09.421523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:09.421523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:09.421523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:09.421523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:09.421523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:09.421523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:09.421523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:35:09.421523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:35:09.421523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:35:09.421523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:35:09.422479 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:35:09.422479 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:35:09.422479 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:35:09.422479 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:35:09.422479 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:09.422479 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:09.422479 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:09.422479 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:09.422479 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:09.422479 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:09.422479 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:09.422479 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:09.422479 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:09.422479 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:09.422479 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:09.422479 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:09.423319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:09.423319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:09.423319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:09.423319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:09.423319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:35:09.423319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:35:09.423319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:35:09.423319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:35:09.423319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:35:09.423319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:35:09.423319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:35:09.423319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:09.426189 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 318/628] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 10 22:35:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:09.427029 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc5481-ff36-4142-940b-0fa80ba2989d None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:09.436558 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 311/629] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:35:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:09.437800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c00ac-a0d8-46d7-b719-c24d652e529e None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:09.493517 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:09.540805 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc5481-ff36-4142-940b-0fa80ba2989d None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:09.553380 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c00ac-a0d8-46d7-b719-c24d652e529e None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:09.579756 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19828a87-e046-4e43-8856-f9f75ef2c6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:09.580198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19828a87-e046-4e43-8856-f9f75ef2c6ea None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:09.580478 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19828a87-e046-4e43-8856-f9f75ef2c6ea None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:09.580752 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19828a87-e046-4e43-8856-f9f75ef2c6ea None admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:09.582085 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19828a87-e046-4e43-8856-f9f75ef2c6ea None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:09.582503 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19828a87-e046-4e43-8856-f9f75ef2c6ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:09.584127 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19828a87-e046-4e43-8856-f9f75ef2c6ea None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:09.593074 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc828f27-6da0-45ce-a65f-152c859e2ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:09.594007 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:09.594849 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc828f27-6da0-45ce-a65f-152c859e2ab5 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:09.595404 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc828f27-6da0-45ce-a65f-152c859e2ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:09.595888 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc828f27-6da0-45ce-a65f-152c859e2ab5 None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:09.596779 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc828f27-6da0-45ce-a65f-152c859e2ab5 None admin] RBAC: Adding query filter params (name=service) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:09.597969 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc828f27-6da0-45ce-a65f-152c859e2ab5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:09.599796 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc828f27-6da0-45ce-a65f-152c859e2ab5 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:09.609312 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 319/630] 10.222.0.76 () {56 vars in 1133 bytes} [Wed Aug 10 22:35:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:09.610032 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 312/631] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 10 22:35:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:09.610307 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19828a87-e046-4e43-8856-f9f75ef2c6ea None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:09.611269 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc828f27-6da0-45ce-a65f-152c859e2ab5 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:09.738667 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc828f27-6da0-45ce-a65f-152c859e2ab5 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:09.743499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19828a87-e046-4e43-8856-f9f75ef2c6ea None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:09.753658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:09.776496 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eac283-83a2-4c16-b199-9202805b90f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:09.777148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eac283-83a2-4c16-b199-9202805b90f7 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:09.777463 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eac283-83a2-4c16-b199-9202805b90f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:09.777683 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eac283-83a2-4c16-b199-9202805b90f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:09.786518 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89cc1d1-36ec-4574-ba5c-375189968868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:09.787484 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cc1d1-36ec-4574-ba5c-375189968868 None admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:09.788001 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cc1d1-36ec-4574-ba5c-375189968868 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:09.788476 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cc1d1-36ec-4574-ba5c-375189968868 None admin] PATH_INFO: `/v3/projects/3caa2ea846f7449b8c483788fc5846c8/users/1e3a76df95a34b9fb9816f22b85d1071/roles/5dfd0f952cd641bbb2257860c4c892ac` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:09.805462 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89cc1d1-36ec-4574-ba5c-375189968868 None admin] RBAC: Authorizing `identity:create_grant(project_id=3caa2ea846f7449b8c483788fc5846c8, user_id=1e3a76df95a34b9fb9816f22b85d1071, role_id=5dfd0f952cd641bbb2257860c4c892ac)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:09.807285 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89cc1d1-36ec-4574-ba5c-375189968868 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:09.829312 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 320/632] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 10 22:35:09 2022] PUT /identity/v3/projects/3caa2ea846f7449b8c483788fc5846c8/users/1e3a76df95a34b9fb9816f22b85d1071/roles/5dfd0f952cd641bbb2257860c4c892ac => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:35:09.829312 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89cc1d1-36ec-4574-ba5c-375189968868 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:09.867680 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eac283-83a2-4c16-b199-9202805b90f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:09.869266 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eac283-83a2-4c16-b199-9202805b90f7 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:09.898052 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:09.956683 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89cc1d1-36ec-4574-ba5c-375189968868 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:09.996360 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267dbb12-550c-4f13-97a5-f8b3ce2c5eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:09.996981 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267dbb12-550c-4f13-97a5-f8b3ce2c5eb1 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:09.997266 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267dbb12-550c-4f13-97a5-f8b3ce2c5eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:09.997560 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267dbb12-550c-4f13-97a5-f8b3ce2c5eb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:09.997845 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 313/633] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:35:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:35:09.999122 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee066c-b8ac-478b-b25d-712daf58f8eb None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:09.999405 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee066c-b8ac-478b-b25d-712daf58f8eb None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:09.999679 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee066c-b8ac-478b-b25d-712daf58f8eb None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:10.001143 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 314/634] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:10.007705 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a2580-14e1-4d77-a478-88ac24bdd9ec None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:10.008078 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a2580-14e1-4d77-a478-88ac24bdd9ec None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:10.008376 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a2580-14e1-4d77-a478-88ac24bdd9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:10.010149 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267dbb12-550c-4f13-97a5-f8b3ce2c5eb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:10.011615 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267dbb12-550c-4f13-97a5-f8b3ce2c5eb1 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:10.027391 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-267dbb12-550c-4f13-97a5-f8b3ce2c5eb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:10.027391 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:10.027391 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:10.027391 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:10.027391 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:10.027391 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:10.027391 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:10.027391 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:10.027391 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:10.027391 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:10.027391 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:10.027391 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:10.027391 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:10.027391 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:10.027391 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:10.027391 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:10.028381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:10.028381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:10.028381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:10.028381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:10.028381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:10.028381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:10.028381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:10.028381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:10.028381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:10.028381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:10.028381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:10.028381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:10.028381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:10.028381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:10.028381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:10.028381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:10.028381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:10.029250 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:10.029250 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:10.029250 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:10.029250 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 321/635] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:10.030372 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267dbb12-550c-4f13-97a5-f8b3ce2c5eb1 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:10.031460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-1d9a2580-14e1-4d77-a478-88ac24bdd9ec None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:10.032731 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-1d9a2580-14e1-4d77-a478-88ac24bdd9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:10.041678 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:10.120703 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 315/636] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:35:10.121827 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a2580-14e1-4d77-a478-88ac24bdd9ec None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:10.142310 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:10.147164 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267dbb12-550c-4f13-97a5-f8b3ce2c5eb1 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:10.196574 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e6250-b1b8-4dea-91c9-3bec254598fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:10.197344 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e6250-b1b8-4dea-91c9-3bec254598fc None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:10.197636 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e6250-b1b8-4dea-91c9-3bec254598fc None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:10.197911 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e6250-b1b8-4dea-91c9-3bec254598fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:10.210363 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241e6250-b1b8-4dea-91c9-3bec254598fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:10.211854 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241e6250-b1b8-4dea-91c9-3bec254598fc None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:10.228303 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-241e6250-b1b8-4dea-91c9-3bec254598fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:10.228303 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:10.228303 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:10.228303 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:10.228303 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:10.228303 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:10.228303 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:10.228303 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:10.228303 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:10.228303 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:10.228303 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:10.228303 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:10.228303 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:10.228303 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:10.228303 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:10.228303 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:10.229609 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:10.229609 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:10.229609 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:10.229609 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:10.229609 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:10.229609 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:10.229609 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:10.229609 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:10.229609 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:10.229609 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:10.229609 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:10.229609 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:10.229609 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:10.229609 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:10.229609 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:10.229609 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:10.229609 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:10.230571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:10.230571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:10.230571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:10.230571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 322/637] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:10.230831 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e6250-b1b8-4dea-91c9-3bec254598fc None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:10.267092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a2580-14e1-4d77-a478-88ac24bdd9ec None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:10.315419 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc913e-45d1-4cc9-8096-4f7fe87f8981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:10.317321 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc913e-45d1-4cc9-8096-4f7fe87f8981 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:10.317920 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc913e-45d1-4cc9-8096-4f7fe87f8981 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:10.318400 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc913e-45d1-4cc9-8096-4f7fe87f8981 None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:10.319247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddc913e-45d1-4cc9-8096-4f7fe87f8981 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:10.320343 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddc913e-45d1-4cc9-8096-4f7fe87f8981 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:10.322420 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddc913e-45d1-4cc9-8096-4f7fe87f8981 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:10.340064 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 316/638] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:10.341221 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc913e-45d1-4cc9-8096-4f7fe87f8981 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:10.345948 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:10.356397 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e6250-b1b8-4dea-91c9-3bec254598fc None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:10.396268 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f320b-cc3f-4005-9a31-29a4e947b287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:10.397266 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f320b-cc3f-4005-9a31-29a4e947b287 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:10.397900 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f320b-cc3f-4005-9a31-29a4e947b287 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:10.398407 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f320b-cc3f-4005-9a31-29a4e947b287 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:10.409502 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1f320b-cc3f-4005-9a31-29a4e947b287 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:35:10.421693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-ab1f320b-cc3f-4005-9a31-29a4e947b287 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:35:10.421693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:10.421693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:10.421693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:10.421693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:10.421693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:10.421693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:10.421693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:10.421693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:10.421693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:10.421693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:10.421693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:10.421693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:35:10.421693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:35:10.421693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:35:10.421693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:35:10.422803 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:35:10.422803 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:35:10.422803 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:35:10.422803 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:35:10.422803 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:10.422803 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:10.422803 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:10.422803 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:10.422803 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:10.422803 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:10.422803 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:10.422803 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:10.422803 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:10.422803 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:10.422803 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:10.422803 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:10.423734 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:10.423734 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:10.423734 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:10.423734 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:10.423734 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:35:10.423734 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:35:10.423734 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:35:10.423734 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:35:10.423734 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:35:10.423734 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:35:10.423734 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:35:10.423734 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:10.431854 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 323/639] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 10 22:35:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:10.432842 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f320b-cc3f-4005-9a31-29a4e947b287 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:10.453188 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:10.469165 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc913e-45d1-4cc9-8096-4f7fe87f8981 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:10.505885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c90060-f243-4995-b327-e960084c99c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:10.506534 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c90060-f243-4995-b327-e960084c99c9 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:10.506814 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c90060-f243-4995-b327-e960084c99c9 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:10.507085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c90060-f243-4995-b327-e960084c99c9 None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:10.507726 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c90060-f243-4995-b327-e960084c99c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:10.508024 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c90060-f243-4995-b327-e960084c99c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:10.509414 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c90060-f243-4995-b327-e960084c99c9 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:10.522048 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 317/640] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:10.522872 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c90060-f243-4995-b327-e960084c99c9 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:10.550573 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f320b-cc3f-4005-9a31-29a4e947b287 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:10.586033 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a504ac-c219-499a-b44e-298d5e6244f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:10.586672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a504ac-c219-499a-b44e-298d5e6244f7 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:10.586955 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a504ac-c219-499a-b44e-298d5e6244f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:10.587234 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a504ac-c219-499a-b44e-298d5e6244f7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:10.593248 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:10.594931 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a504ac-c219-499a-b44e-298d5e6244f7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:10.596408 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a504ac-c219-499a-b44e-298d5e6244f7 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:10.607625 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-c1a504ac-c219-499a-b44e-298d5e6244f7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 22:35:10.607625 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:10.607625 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:10.607625 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:10.607625 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:10.607625 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:10.607625 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:10.607625 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:10.607625 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:10.607625 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:10.607625 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:10.607625 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:10.607625 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:35:10.607625 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:35:10.607625 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:35:10.607625 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:35:10.608651 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:10.608651 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:10.608651 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:35:10.608651 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:10.608651 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:35:10.608651 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:10.608651 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:10.608651 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:10.608651 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:10.608651 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:10.608651 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:10.608651 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:10.608651 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:10.608651 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:10.608651 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:10.608651 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:10.608651 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:10.608651 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:10.609674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:10.609674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:10.609674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:35:10.609674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:35:10.609674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:35:10.609674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:35:10.609674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:35:10.609674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:35:10.609674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 22:35:10.609674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:10.609674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 324/641] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:35:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:10.610408 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8454d-e4c8-47fa-a437-df94bdd76828 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:10.610698 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8454d-e4c8-47fa-a437-df94bdd76828 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:10.610973 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8454d-e4c8-47fa-a437-df94bdd76828 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:10.615227 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 325/642] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:10.615227 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8454d-e4c8-47fa-a437-df94bdd76828 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:10.643688 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c90060-f243-4995-b327-e960084c99c9 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:10.683010 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74365334-1d41-4c9a-a520-10e2b2ee1041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:10.683664 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74365334-1d41-4c9a-a520-10e2b2ee1041 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:10.683956 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74365334-1d41-4c9a-a520-10e2b2ee1041 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:10.684217 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74365334-1d41-4c9a-a520-10e2b2ee1041 None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:10.684869 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74365334-1d41-4c9a-a520-10e2b2ee1041 None admin] RBAC: Adding query filter params (name=service) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:10.685170 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74365334-1d41-4c9a-a520-10e2b2ee1041 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:10.686581 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74365334-1d41-4c9a-a520-10e2b2ee1041 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:10.687788 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:10.701249 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 318/643] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 10 22:35:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:10.701249 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74365334-1d41-4c9a-a520-10e2b2ee1041 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:10.727183 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8454d-e4c8-47fa-a437-df94bdd76828 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:10.761870 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b79e32-d9b2-412e-963f-b32e3bb56b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:10.762505 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b79e32-d9b2-412e-963f-b32e3bb56b5d None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:10.762786 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b79e32-d9b2-412e-963f-b32e3bb56b5d None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:10.763061 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b79e32-d9b2-412e-963f-b32e3bb56b5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:10.770256 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b79e32-d9b2-412e-963f-b32e3bb56b5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:10.771755 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b79e32-d9b2-412e-963f-b32e3bb56b5d None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:10.787784 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-29b79e32-d9b2-412e-963f-b32e3bb56b5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:10.787784 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:10.787784 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:10.787784 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:10.787784 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:10.787784 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:10.787784 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:10.787784 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:10.787784 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:10.787784 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:10.787784 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:10.787784 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:10.787784 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:35:10.787784 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:35:10.787784 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:35:10.788914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:35:10.788914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:10.788914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:10.788914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:10.788914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:10.788914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:10.788914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:10.788914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:10.788914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:10.788914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:10.788914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:10.788914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:10.788914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:10.788914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:10.788914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:10.788914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:10.788914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:10.790103 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:35:10.790103 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:35:10.790103 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:35:10.790103 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:35:10.790103 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:35:10.790103 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:35:10.790103 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:10.790103 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:10.790103 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 326/644] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:35:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:10.791513 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44516fd-85c4-421a-bfe3-e8f5b1ab4aa8 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:10.791917 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44516fd-85c4-421a-bfe3-e8f5b1ab4aa8 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:10.792328 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44516fd-85c4-421a-bfe3-e8f5b1ab4aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:10.814308 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-b44516fd-85c4-421a-bfe3-e8f5b1ab4aa8 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:10.815519 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-b44516fd-85c4-421a-bfe3-e8f5b1ab4aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:10.825823 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74365334-1d41-4c9a-a520-10e2b2ee1041 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:10.850980 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:10.863574 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e717af3-45f7-415e-9b06-95bfb80dc7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:10.864188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e717af3-45f7-415e-9b06-95bfb80dc7fa None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:10.864467 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e717af3-45f7-415e-9b06-95bfb80dc7fa None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:10.864733 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e717af3-45f7-415e-9b06-95bfb80dc7fa None admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:10.865476 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e717af3-45f7-415e-9b06-95bfb80dc7fa None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:10.865774 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e717af3-45f7-415e-9b06-95bfb80dc7fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:10.867223 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e717af3-45f7-415e-9b06-95bfb80dc7fa None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:10.888871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 319/645] 10.222.0.76 () {56 vars in 1137 bytes} [Wed Aug 10 22:35:10 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:10.888871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e717af3-45f7-415e-9b06-95bfb80dc7fa None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:10.893169 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 327/646] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:35:10.898227 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44516fd-85c4-421a-bfe3-e8f5b1ab4aa8 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:10.967372 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:10.994539 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e717af3-45f7-415e-9b06-95bfb80dc7fa None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:11.011931 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44516fd-85c4-421a-bfe3-e8f5b1ab4aa8 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:11.033096 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0af193-25da-46f4-9dd6-45668742a804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:11.033774 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0af193-25da-46f4-9dd6-45668742a804 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:11.034057 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0af193-25da-46f4-9dd6-45668742a804 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:11.034330 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0af193-25da-46f4-9dd6-45668742a804 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:11.046002 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0af193-25da-46f4-9dd6-45668742a804 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:11.047617 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0af193-25da-46f4-9dd6-45668742a804 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:11.049107 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0477bc9-087c-4fc2-9d79-2ee1ccc3dc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:11.049761 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0477bc9-087c-4fc2-9d79-2ee1ccc3dc74 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:11.050042 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0477bc9-087c-4fc2-9d79-2ee1ccc3dc74 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:11.051124 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0477bc9-087c-4fc2-9d79-2ee1ccc3dc74 None admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:11.052095 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0477bc9-087c-4fc2-9d79-2ee1ccc3dc74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:11.052403 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0477bc9-087c-4fc2-9d79-2ee1ccc3dc74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:11.053942 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0477bc9-087c-4fc2-9d79-2ee1ccc3dc74 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:11.069716 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 328/647] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:35:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:11.071105 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0477bc9-087c-4fc2-9d79-2ee1ccc3dc74 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:11.185658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-dd0af193-25da-46f4-9dd6-45668742a804 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:11.185658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:11.185658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:11.185658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:11.185658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:11.185658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:11.185658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:11.185658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:11.185658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:11.185658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:11.185658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:11.185658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:11.185658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:11.185658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:11.185658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:11.185658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:11.186687 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:11.186687 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:11.186687 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:11.186687 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:11.186687 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:11.186687 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:11.186687 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:11.186687 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:11.186687 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:11.186687 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:11.186687 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:11.186687 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:11.186687 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:11.186687 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:11.186687 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:11.186687 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:11.186687 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:11.187571 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:11.187571 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:11.187571 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:11.187571 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 320/648] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:11.187571 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0af193-25da-46f4-9dd6-45668742a804 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:11.188494 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0477bc9-087c-4fc2-9d79-2ee1ccc3dc74 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:11.215304 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:11.226887 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:11.232167 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae40bb-95ca-4ca9-9bfc-a8554afa1809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:11.233373 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae40bb-95ca-4ca9-9bfc-a8554afa1809 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:11.233740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae40bb-95ca-4ca9-9bfc-a8554afa1809 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:11.234123 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae40bb-95ca-4ca9-9bfc-a8554afa1809 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:11.301528 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0af193-25da-46f4-9dd6-45668742a804 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:11.314680 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae40bb-95ca-4ca9-9bfc-a8554afa1809 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:11.316883 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae40bb-95ca-4ca9-9bfc-a8554afa1809 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:11.338732 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5bc85a-5586-43d6-a48b-d3d79e5b5937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:11.339374 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bc85a-5586-43d6-a48b-d3d79e5b5937 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:11.339657 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bc85a-5586-43d6-a48b-d3d79e5b5937 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:11.339924 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bc85a-5586-43d6-a48b-d3d79e5b5937 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:11.351618 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5bc85a-5586-43d6-a48b-d3d79e5b5937 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:35:11.356261 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-dc5bc85a-5586-43d6-a48b-d3d79e5b5937 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:35:11.356261 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:11.356261 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:11.356261 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:11.356261 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:11.356261 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:11.356261 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:11.356261 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:11.356261 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:11.356261 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:11.356261 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:11.356261 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:11.356261 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:35:11.356261 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:35:11.356261 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:35:11.356261 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:35:11.357201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:35:11.357201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:35:11.357201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:35:11.357201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:35:11.357201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:11.357201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:11.357201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:11.357201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:11.357201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:11.357201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:11.357201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:11.357201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:11.357201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:11.357201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:11.357201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:11.357201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:11.358865 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:11.358865 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:11.358865 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:11.358865 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:11.358865 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:35:11.358865 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:35:11.358865 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:35:11.358865 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:35:11.358865 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:35:11.358865 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:35:11.358865 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:35:11.358865 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:11.358865 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 321/649] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 10 22:35:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:11.358865 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5bc85a-5586-43d6-a48b-d3d79e5b5937 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:11.433357 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 329/650] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:35:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:35:11.434268 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae40bb-95ca-4ca9-9bfc-a8554afa1809 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:11.461142 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:11.471536 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:11.479325 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5bc85a-5586-43d6-a48b-d3d79e5b5937 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:11.516226 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca6726b-6eeb-49d6-89a7-f4a969f5f16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:11.516844 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6726b-6eeb-49d6-89a7-f4a969f5f16a None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:11.517115 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6726b-6eeb-49d6-89a7-f4a969f5f16a None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:11.517417 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6726b-6eeb-49d6-89a7-f4a969f5f16a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:11.518140 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca6726b-6eeb-49d6-89a7-f4a969f5f16a None admin] RBAC: Adding query filter params (role.id=5dfd0f952cd641bbb2257860c4c892ac, scope.project.id=3caa2ea846f7449b8c483788fc5846c8, user.id=b22838a8ad3949e59cdbaaa1801cd07f) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:11.518461 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca6726b-6eeb-49d6-89a7-f4a969f5f16a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:11.520374 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca6726b-6eeb-49d6-89a7-f4a969f5f16a None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:11.526341 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 322/651] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:35:11 2022] GET /identity/v3/role_assignments?user.id=b22838a8ad3949e59cdbaaa1801cd07f&scope.project.id=3caa2ea846f7449b8c483788fc5846c8&role.id=5dfd0f952cd641bbb2257860c4c892ac => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:11.526988 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca6726b-6eeb-49d6-89a7-f4a969f5f16a None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:11.573758 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae40bb-95ca-4ca9-9bfc-a8554afa1809 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:11.616422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beac016e-3b86-4161-a202-2f1349a31bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:11.617052 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac016e-3b86-4161-a202-2f1349a31bc2 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:11.617372 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac016e-3b86-4161-a202-2f1349a31bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:11.617708 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac016e-3b86-4161-a202-2f1349a31bc2 None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:11.618367 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beac016e-3b86-4161-a202-2f1349a31bc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:11.618844 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beac016e-3b86-4161-a202-2f1349a31bc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:11.620352 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beac016e-3b86-4161-a202-2f1349a31bc2 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:11.634955 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 330/652] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:11.635825 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beac016e-3b86-4161-a202-2f1349a31bc2 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:11.678594 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca6726b-6eeb-49d6-89a7-f4a969f5f16a None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:11.732045 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7337df36-f67d-4678-8a33-93367954d7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:11.733030 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337df36-f67d-4678-8a33-93367954d7ae None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:11.733581 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337df36-f67d-4678-8a33-93367954d7ae None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:11.733922 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337df36-f67d-4678-8a33-93367954d7ae None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:11.734663 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7337df36-f67d-4678-8a33-93367954d7ae None admin] RBAC: Adding query filter params (name=service) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:11.734974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7337df36-f67d-4678-8a33-93367954d7ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:11.736493 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7337df36-f67d-4678-8a33-93367954d7ae None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:11.745678 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:11.747166 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 323/653] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 10 22:35:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:11.749076 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7337df36-f67d-4678-8a33-93367954d7ae None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:11.777132 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:11.780028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beac016e-3b86-4161-a202-2f1349a31bc2 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:11.823972 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba653ed-22fa-4757-affa-1c32a8437536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:11.824862 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba653ed-22fa-4757-affa-1c32a8437536 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:11.825190 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba653ed-22fa-4757-affa-1c32a8437536 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:11.825492 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba653ed-22fa-4757-affa-1c32a8437536 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:11.839426 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba653ed-22fa-4757-affa-1c32a8437536 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:11.841536 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba653ed-22fa-4757-affa-1c32a8437536 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:11.858243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-8ba653ed-22fa-4757-affa-1c32a8437536 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:11.858243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:11.858243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:11.858243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:11.858243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:11.858243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:11.858243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:11.858243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:11.858243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:11.858243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:11.858243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:11.858243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:11.858243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:11.858243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:11.858243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:11.858243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:11.859399 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:11.859399 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:11.859399 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:11.859399 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:11.859399 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:11.859399 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:11.859399 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:11.859399 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:11.859399 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:11.859399 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:11.859399 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:11.859399 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:11.859399 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:11.859399 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:11.859399 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:11.859399 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:11.859399 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:11.860723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:11.860723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:11.860723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:11.860723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 331/654] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:11.860723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba653ed-22fa-4757-affa-1c32a8437536 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:11.876674 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7337df36-f67d-4678-8a33-93367954d7ae None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:11.919675 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018733a-331f-4564-9e70-a0d8ba7b945a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:11.920327 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018733a-331f-4564-9e70-a0d8ba7b945a None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:11.920591 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018733a-331f-4564-9e70-a0d8ba7b945a None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:11.920865 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018733a-331f-4564-9e70-a0d8ba7b945a None admin] PATH_INFO: `/v3/users/glance` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:11.928568 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f018733a-331f-4564-9e70-a0d8ba7b945a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:11.930048 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f018733a-331f-4564-9e70-a0d8ba7b945a None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:11.942151 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-f018733a-331f-4564-9e70-a0d8ba7b945a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 22:35:11.942151 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:11.942151 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:11.942151 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:11.942151 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:11.942151 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:11.942151 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:11.942151 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:11.942151 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:11.942151 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:11.942151 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:11.942151 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:11.942151 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:35:11.942151 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:35:11.942151 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:35:11.942151 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:35:11.943311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:11.943311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:11.943311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:35:11.943311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:11.943311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:35:11.943311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:11.943311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:11.943311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:11.943311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:11.943311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:11.943311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:11.943311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:11.943311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:11.943311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:11.943311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:11.943311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:11.943311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:11.943311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:11.944455 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:11.944455 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:11.944455 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:35:11.944455 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:35:11.944455 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:35:11.944455 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:35:11.944455 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:35:11.944455 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:35:11.944455 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 22:35:11.944455 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:11.944455 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 324/655] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:35:11 2022] GET /identity/v3/users/glance => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:11.944455 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018733a-331f-4564-9e70-a0d8ba7b945a None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:12.002123 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba653ed-22fa-4757-affa-1c32a8437536 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:12.032570 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:12.041059 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e1c04-a441-4bc5-bed0-6de579cf2e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:12.041728 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e1c04-a441-4bc5-bed0-6de579cf2e37 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:12.042008 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e1c04-a441-4bc5-bed0-6de579cf2e37 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:12.042286 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e1c04-a441-4bc5-bed0-6de579cf2e37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:12.044198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:12.054189 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5e1c04-a441-4bc5-bed0-6de579cf2e37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:12.055650 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5e1c04-a441-4bc5-bed0-6de579cf2e37 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:12.056702 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018733a-331f-4564-9e70-a0d8ba7b945a None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:12.070326 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-cc5e1c04-a441-4bc5-bed0-6de579cf2e37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:12.070326 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:12.070326 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:12.070326 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:12.070326 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:12.070326 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:12.070326 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:12.070326 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:12.070326 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:12.070326 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:12.070326 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:12.070326 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:12.070326 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:12.070326 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:12.070326 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:12.070326 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:12.071293 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:12.071293 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:12.071293 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:12.071293 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:12.071293 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:12.071293 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:12.071293 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:12.071293 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:12.071293 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:12.071293 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:12.071293 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:12.071293 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:12.071293 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:12.071293 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:12.071293 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:12.071293 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:12.071293 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:12.072184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:12.072184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:12.072184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:12.072184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 332/656] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:12.072184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e1c04-a441-4bc5-bed0-6de579cf2e37 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:12.107040 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54158128-b5be-4618-9aa8-a568731e88a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:12.107661 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54158128-b5be-4618-9aa8-a568731e88a7 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:12.107947 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54158128-b5be-4618-9aa8-a568731e88a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:12.108225 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54158128-b5be-4618-9aa8-a568731e88a7 None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:12.108869 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54158128-b5be-4618-9aa8-a568731e88a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:12.109176 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54158128-b5be-4618-9aa8-a568731e88a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:12.110596 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54158128-b5be-4618-9aa8-a568731e88a7 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:12.130311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 325/657] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:12.130311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912bf6b2-5bdb-4083-ac85-8a6e2b6670eb None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:12.130311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912bf6b2-5bdb-4083-ac85-8a6e2b6670eb None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:12.130311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912bf6b2-5bdb-4083-ac85-8a6e2b6670eb None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:12.133213 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 326/658] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:12.133541 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912bf6b2-5bdb-4083-ac85-8a6e2b6670eb None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:12.193372 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e1c04-a441-4bc5-bed0-6de579cf2e37 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:12.234554 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8272b27c-b753-41f9-b6f2-5e2806fb777b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:12.235550 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272b27c-b753-41f9-b6f2-5e2806fb777b None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:12.235832 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272b27c-b753-41f9-b6f2-5e2806fb777b None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:12.236105 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272b27c-b753-41f9-b6f2-5e2806fb777b None admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:12.237094 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8272b27c-b753-41f9-b6f2-5e2806fb777b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:12.238227 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8272b27c-b753-41f9-b6f2-5e2806fb777b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:12.240838 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8272b27c-b753-41f9-b6f2-5e2806fb777b None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:12.247305 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912bf6b2-5bdb-4083-ac85-8a6e2b6670eb None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:12.265524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 333/659] 10.222.0.76 () {56 vars in 1137 bytes} [Wed Aug 10 22:35:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:12.266277 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8272b27c-b753-41f9-b6f2-5e2806fb777b None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:12.291160 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396727cd-e392-4de6-9d67-1f7ed860b2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:12.292115 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396727cd-e392-4de6-9d67-1f7ed860b2ab None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:12.292504 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396727cd-e392-4de6-9d67-1f7ed860b2ab None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:12.292885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396727cd-e392-4de6-9d67-1f7ed860b2ab None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:12.293646 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396727cd-e392-4de6-9d67-1f7ed860b2ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:12.294044 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396727cd-e392-4de6-9d67-1f7ed860b2ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:12.295501 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396727cd-e392-4de6-9d67-1f7ed860b2ab None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:12.305110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:12.313628 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 327/660] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:12.315110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b600e0-f044-4b34-bc3a-0adc4c6ab529 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:12.315383 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b600e0-f044-4b34-bc3a-0adc4c6ab529 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:12.315821 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b600e0-f044-4b34-bc3a-0adc4c6ab529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:12.321288 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:12.334345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-26b600e0-f044-4b34-bc3a-0adc4c6ab529 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:12.335565 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-26b600e0-f044-4b34-bc3a-0adc4c6ab529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:12.385671 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8272b27c-b753-41f9-b6f2-5e2806fb777b None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:12.409915 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 328/661] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:35:12.409915 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b600e0-f044-4b34-bc3a-0adc4c6ab529 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:12.454674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f93c75c-2954-4e7a-b9c7-b488ac2a8a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:12.455641 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93c75c-2954-4e7a-b9c7-b488ac2a8a50 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:12.456019 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93c75c-2954-4e7a-b9c7-b488ac2a8a50 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:12.456565 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93c75c-2954-4e7a-b9c7-b488ac2a8a50 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:12.466677 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f93c75c-2954-4e7a-b9c7-b488ac2a8a50 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:12.469036 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f93c75c-2954-4e7a-b9c7-b488ac2a8a50 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:12.487095 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-7f93c75c-2954-4e7a-b9c7-b488ac2a8a50 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:12.487095 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:12.487095 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:12.487095 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:12.487095 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:12.487095 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:12.487095 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:12.487095 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:12.487095 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:12.487095 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:12.487095 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:12.487095 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:12.487095 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:35:12.487095 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:35:12.487095 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:35:12.489626 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:35:12.489626 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:12.489626 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:12.489626 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:12.489626 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:12.489626 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:12.489626 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:12.489626 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:12.489626 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:12.489626 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:12.489626 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:12.489626 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:12.489626 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:12.489626 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:12.489626 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:12.489626 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:12.489626 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:12.491419 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:35:12.491419 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:35:12.491419 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:35:12.491419 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:35:12.491419 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:35:12.491419 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:35:12.491419 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:12.491419 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:12.491419 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 334/662] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:35:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:12.491419 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f93c75c-2954-4e7a-b9c7-b488ac2a8a50 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:12.566943 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b600e0-f044-4b34-bc3a-0adc4c6ab529 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:12.614405 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:12.622289 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d0bba-301c-49bb-9c43-21e1f75e54a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:12.623247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d0bba-301c-49bb-9c43-21e1f75e54a7 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:12.623862 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d0bba-301c-49bb-9c43-21e1f75e54a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:12.624674 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d0bba-301c-49bb-9c43-21e1f75e54a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:12.648225 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f93c75c-2954-4e7a-b9c7-b488ac2a8a50 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:12.652600 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:12.703437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440fddb3-298c-4a42-aa76-968bf8d1a016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:12.704402 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440fddb3-298c-4a42-aa76-968bf8d1a016 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:12.704866 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440fddb3-298c-4a42-aa76-968bf8d1a016 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:12.705355 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440fddb3-298c-4a42-aa76-968bf8d1a016 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:12.716927 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440fddb3-298c-4a42-aa76-968bf8d1a016 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:12.719291 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440fddb3-298c-4a42-aa76-968bf8d1a016 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:12.734172 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d0bba-301c-49bb-9c43-21e1f75e54a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:12.736993 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d0bba-301c-49bb-9c43-21e1f75e54a7 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:12.739610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-440fddb3-298c-4a42-aa76-968bf8d1a016 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 22:35:12.739610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:12.739610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:12.739610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:12.739610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:12.739610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:12.739610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:12.739610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:12.739610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:12.739610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:12.739610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:12.739610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:12.739610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:35:12.739610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:35:12.739610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:35:12.739610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:35:12.748728 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:12.748728 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:12.748728 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:35:12.748728 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:12.748728 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:35:12.748728 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:12.748728 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:12.748728 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:12.748728 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:12.748728 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:12.748728 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:12.748728 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:12.748728 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:12.748728 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:12.748728 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:12.748728 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:12.748728 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:12.748728 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:12.750598 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:12.750598 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:12.750598 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:35:12.750598 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:35:12.750598 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:35:12.750598 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:35:12.750598 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:35:12.750598 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:35:12.750598 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 22:35:12.750598 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:12.750598 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 335/663] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 10 22:35:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:12.750598 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440fddb3-298c-4a42-aa76-968bf8d1a016 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:12.858270 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440fddb3-298c-4a42-aa76-968bf8d1a016 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:12.861051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 329/664] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:35:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:35:12.862131 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d0bba-301c-49bb-9c43-21e1f75e54a7 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:12.896427 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594faaf4-c88b-4db8-837c-ef9ae837be08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:12.897056 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594faaf4-c88b-4db8-837c-ef9ae837be08 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:12.897357 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594faaf4-c88b-4db8-837c-ef9ae837be08 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:12.897629 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594faaf4-c88b-4db8-837c-ef9ae837be08 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:12.901605 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:12.908097 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594faaf4-c88b-4db8-837c-ef9ae837be08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:35:12.913686 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-594faaf4-c88b-4db8-837c-ef9ae837be08 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:35:12.913686 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:12.913686 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:12.913686 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:12.913686 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:12.913686 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:12.913686 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:12.913686 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:12.913686 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:12.913686 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:12.913686 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:12.913686 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:12.913686 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:35:12.913686 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:35:12.913686 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:35:12.913686 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:35:12.914672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:35:12.914672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:35:12.914672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:35:12.914672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:35:12.914672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:12.914672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:12.914672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:12.914672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:12.914672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:12.914672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:12.914672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:12.914672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:12.914672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:12.914672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:12.914672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:12.914672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:12.915556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:12.915556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:12.915556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:12.915556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:12.915556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:35:12.915556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:35:12.915556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:35:12.915556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:35:12.915556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:35:12.915556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:35:12.915556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:35:12.915556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:12.915556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 336/665] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 10 22:35:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:12.915556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594faaf4-c88b-4db8-837c-ef9ae837be08 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:12.947530 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:12.982092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d0bba-301c-49bb-9c43-21e1f75e54a7 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:13.017578 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea6a83-015e-4ace-8afc-dcb0b004b8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:13.018182 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea6a83-015e-4ace-8afc-dcb0b004b8d9 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:13.018460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea6a83-015e-4ace-8afc-dcb0b004b8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:13.019459 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea6a83-015e-4ace-8afc-dcb0b004b8d9 None admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:13.020100 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ea6a83-015e-4ace-8afc-dcb0b004b8d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:13.020424 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ea6a83-015e-4ace-8afc-dcb0b004b8d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:13.021882 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ea6a83-015e-4ace-8afc-dcb0b004b8d9 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:13.028430 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594faaf4-c88b-4db8-837c-ef9ae837be08 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:13.037582 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 330/666] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:35:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:13.038290 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea6a83-015e-4ace-8afc-dcb0b004b8d9 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:13.065704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34004252-5895-47f4-aa9d-70062386ba06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:13.066349 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34004252-5895-47f4-aa9d-70062386ba06 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:13.066644 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34004252-5895-47f4-aa9d-70062386ba06 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:13.066858 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34004252-5895-47f4-aa9d-70062386ba06 None admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:13.067717 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34004252-5895-47f4-aa9d-70062386ba06 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:13.068035 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34004252-5895-47f4-aa9d-70062386ba06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:13.069470 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34004252-5895-47f4-aa9d-70062386ba06 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:13.089529 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 337/667] 10.222.0.76 () {56 vars in 1139 bytes} [Wed Aug 10 22:35:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:13.090176 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34004252-5895-47f4-aa9d-70062386ba06 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:13.138011 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:13.149077 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea6a83-015e-4ace-8afc-dcb0b004b8d9 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:13.178442 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:13.185421 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f6b91-54bd-4d76-bfde-1023a8211720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:13.186061 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f6b91-54bd-4d76-bfde-1023a8211720 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:13.186345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f6b91-54bd-4d76-bfde-1023a8211720 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:13.186612 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f6b91-54bd-4d76-bfde-1023a8211720 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:13.197321 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11f6b91-54bd-4d76-bfde-1023a8211720 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:13.198783 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11f6b91-54bd-4d76-bfde-1023a8211720 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:13.200857 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34004252-5895-47f4-aa9d-70062386ba06 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:13.213692 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-f11f6b91-54bd-4d76-bfde-1023a8211720 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:13.213692 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:13.213692 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:13.213692 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:13.213692 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:13.213692 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:13.213692 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:13.213692 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:13.213692 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:13.213692 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:13.213692 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:13.213692 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:13.213692 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:13.213692 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:13.213692 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:13.213692 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:13.214838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:13.214838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:13.214838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:13.214838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:13.214838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:13.214838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:13.214838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:13.214838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:13.214838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:13.214838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:13.214838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:13.214838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:13.214838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:13.214838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:13.214838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:13.214838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:13.214838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:13.215737 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:13.215737 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:13.215737 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:13.215737 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 331/668] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:13.215737 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f6b91-54bd-4d76-bfde-1023a8211720 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:13.246027 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875caea7-3166-4ed6-9afd-652064cddaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:13.246962 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875caea7-3166-4ed6-9afd-652064cddaed None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:13.247515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875caea7-3166-4ed6-9afd-652064cddaed None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:13.247792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875caea7-3166-4ed6-9afd-652064cddaed None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:13.248606 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875caea7-3166-4ed6-9afd-652064cddaed None admin] RBAC: Adding query filter params (name=service) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:13.248911 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875caea7-3166-4ed6-9afd-652064cddaed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:13.250896 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875caea7-3166-4ed6-9afd-652064cddaed None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:13.263410 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 338/669] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 10 22:35:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:13.264574 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875caea7-3166-4ed6-9afd-652064cddaed None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:13.344717 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f6b91-54bd-4d76-bfde-1023a8211720 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:13.376396 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875caea7-3166-4ed6-9afd-652064cddaed None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:13.380974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad5fc0d-132c-446c-b28a-045d3a0fa97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:13.381615 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5fc0d-132c-446c-b28a-045d3a0fa97a None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:13.381894 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5fc0d-132c-446c-b28a-045d3a0fa97a None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:13.382168 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5fc0d-132c-446c-b28a-045d3a0fa97a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:13.382979 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad5fc0d-132c-446c-b28a-045d3a0fa97a None admin] RBAC: Adding query filter params (role.id=5dfd0f952cd641bbb2257860c4c892ac, scope.project.id=3caa2ea846f7449b8c483788fc5846c8, user.id=a7f0dd6c9e2d4315966e90cd3b927b13) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:13.383281 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad5fc0d-132c-446c-b28a-045d3a0fa97a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:13.384707 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad5fc0d-132c-446c-b28a-045d3a0fa97a None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:13.390848 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 332/670] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:35:13 2022] GET /identity/v3/role_assignments?user.id=a7f0dd6c9e2d4315966e90cd3b927b13&scope.project.id=3caa2ea846f7449b8c483788fc5846c8&role.id=5dfd0f952cd641bbb2257860c4c892ac => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:13.391587 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad5fc0d-132c-446c-b28a-045d3a0fa97a None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:13.401993 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:13.435932 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cab31e-0ef5-4ff8-8dba-555f8565ca17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:13.436915 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cab31e-0ef5-4ff8-8dba-555f8565ca17 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:13.437451 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cab31e-0ef5-4ff8-8dba-555f8565ca17 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:13.437955 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cab31e-0ef5-4ff8-8dba-555f8565ca17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:13.460894 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:13.521385 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad5fc0d-132c-446c-b28a-045d3a0fa97a None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:13.542183 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cab31e-0ef5-4ff8-8dba-555f8565ca17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:13.544043 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cab31e-0ef5-4ff8-8dba-555f8565ca17 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:13.564566 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc3a22-9ee0-4409-8a48-b8338c351a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:13.565533 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc3a22-9ee0-4409-8a48-b8338c351a04 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:13.565813 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc3a22-9ee0-4409-8a48-b8338c351a04 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:13.566085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc3a22-9ee0-4409-8a48-b8338c351a04 None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:13.566730 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc3a22-9ee0-4409-8a48-b8338c351a04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:13.567027 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc3a22-9ee0-4409-8a48-b8338c351a04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:13.568927 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc3a22-9ee0-4409-8a48-b8338c351a04 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:13.586017 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 333/671] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:13.586884 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc3a22-9ee0-4409-8a48-b8338c351a04 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:13.666441 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:13.689304 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 339/672] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:35:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:35:13.689304 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cab31e-0ef5-4ff8-8dba-555f8565ca17 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:13.712187 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc3a22-9ee0-4409-8a48-b8338c351a04 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:13.740457 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:13.750374 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f035e46-37d9-4754-8b93-d34339aff54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:13.751009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f035e46-37d9-4754-8b93-d34339aff54a None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:13.751285 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f035e46-37d9-4754-8b93-d34339aff54a None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:13.751551 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f035e46-37d9-4754-8b93-d34339aff54a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:13.762039 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f035e46-37d9-4754-8b93-d34339aff54a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:13.763479 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f035e46-37d9-4754-8b93-d34339aff54a None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:13.781351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-6f035e46-37d9-4754-8b93-d34339aff54a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:13.781351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:13.781351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:13.781351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:13.781351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:13.781351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:13.781351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:13.781351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:13.781351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:13.781351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:13.781351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:13.781351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:13.781351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:13.781351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:13.781351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:13.781351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:13.782461 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:13.782461 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:13.782461 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:13.782461 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:13.782461 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:13.782461 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:13.782461 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:13.782461 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:13.782461 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:13.782461 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:13.782461 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:13.782461 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:13.782461 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:13.782461 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:13.782461 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:13.782461 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:13.782461 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:13.783365 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:13.783365 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:13.783365 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:13.783365 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 334/673] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:13.783365 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f035e46-37d9-4754-8b93-d34339aff54a None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:13.804594 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cab31e-0ef5-4ff8-8dba-555f8565ca17 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:13.845024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e37906e-ecbd-452c-bc2e-bdec2178fe08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:13.845682 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37906e-ecbd-452c-bc2e-bdec2178fe08 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:13.845962 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37906e-ecbd-452c-bc2e-bdec2178fe08 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:13.846238 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37906e-ecbd-452c-bc2e-bdec2178fe08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:13.854187 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e37906e-ecbd-452c-bc2e-bdec2178fe08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:13.855641 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e37906e-ecbd-452c-bc2e-bdec2178fe08 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:13.867875 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-8e37906e-ecbd-452c-bc2e-bdec2178fe08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:13.867875 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:13.867875 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:13.867875 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:13.867875 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:13.867875 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:13.867875 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:13.867875 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:13.867875 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:13.867875 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:13.867875 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:13.867875 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:13.867875 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:35:13.867875 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:35:13.867875 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:35:13.868807 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:35:13.868807 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:13.868807 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:13.868807 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:13.868807 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:13.868807 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:13.868807 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:13.868807 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:13.868807 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:13.868807 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:13.868807 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:13.868807 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:13.868807 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:13.868807 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:13.868807 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:13.868807 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:13.868807 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:13.869750 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:35:13.869750 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:35:13.869750 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:35:13.869750 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:35:13.869750 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:35:13.869750 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:35:13.869750 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:13.869750 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:13.873623 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 340/674] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:35:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:13.875445 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a825426-3a89-45b8-ac65-952df3772290 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:13.884141 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a825426-3a89-45b8-ac65-952df3772290 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:13.884141 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a825426-3a89-45b8-ac65-952df3772290 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:13.884141 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 341/675] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:13.884141 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a825426-3a89-45b8-ac65-952df3772290 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:13.903261 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f035e46-37d9-4754-8b93-d34339aff54a None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:13.921728 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:13.943097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a179ab1f-626b-4fb6-93d2-6f6f06756817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:13.943605 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179ab1f-626b-4fb6-93d2-6f6f06756817 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:13.943890 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179ab1f-626b-4fb6-93d2-6f6f06756817 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:13.944211 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179ab1f-626b-4fb6-93d2-6f6f06756817 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:13.955479 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a179ab1f-626b-4fb6-93d2-6f6f06756817 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:13.957876 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a179ab1f-626b-4fb6-93d2-6f6f06756817 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:13.973612 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-a179ab1f-626b-4fb6-93d2-6f6f06756817 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:13.973612 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:13.973612 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:13.973612 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:13.973612 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:13.973612 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:13.973612 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:13.973612 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:13.973612 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:13.973612 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:13.973612 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:13.973612 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:13.973612 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:13.973612 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:13.973612 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:13.973612 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:13.974624 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:13.974624 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:13.974624 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:13.974624 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:13.974624 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:13.974624 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:13.974624 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:13.974624 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:13.974624 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:13.974624 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:13.974624 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:13.974624 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:13.974624 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:13.974624 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:13.974624 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:13.974624 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:13.974624 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:13.975538 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:13.975538 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:13.975538 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:13.979399 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 335/676] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:13.979996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a179ab1f-626b-4fb6-93d2-6f6f06756817 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:14.000549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a825426-3a89-45b8-ac65-952df3772290 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:14.003941 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:14.043425 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f60125-bb1f-4120-a9e8-6b8b68e5fbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:14.044195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f60125-bb1f-4120-a9e8-6b8b68e5fbac None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:14.044475 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f60125-bb1f-4120-a9e8-6b8b68e5fbac None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:14.045070 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f60125-bb1f-4120-a9e8-6b8b68e5fbac None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:14.045744 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f60125-bb1f-4120-a9e8-6b8b68e5fbac None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:14.046048 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f60125-bb1f-4120-a9e8-6b8b68e5fbac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:14.047424 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f60125-bb1f-4120-a9e8-6b8b68e5fbac None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:14.063210 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 342/677] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:14.063210 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475a127-4f9e-4885-beaf-ba95bd49473f None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:14.063210 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475a127-4f9e-4885-beaf-ba95bd49473f None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:14.063210 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475a127-4f9e-4885-beaf-ba95bd49473f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:14.078917 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-d475a127-4f9e-4885-beaf-ba95bd49473f None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:14.080215 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-d475a127-4f9e-4885-beaf-ba95bd49473f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:14.093425 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a179ab1f-626b-4fb6-93d2-6f6f06756817 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:14.131618 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7aceb-a9bc-42b4-ba42-724a77fd77fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:14.132675 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7aceb-a9bc-42b4-ba42-724a77fd77fe None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:14.133097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7aceb-a9bc-42b4-ba42-724a77fd77fe None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:14.133414 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7aceb-a9bc-42b4-ba42-724a77fd77fe None admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:14.134107 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f7aceb-a9bc-42b4-ba42-724a77fd77fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:14.134419 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f7aceb-a9bc-42b4-ba42-724a77fd77fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:14.135852 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f7aceb-a9bc-42b4-ba42-724a77fd77fe None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:14.153783 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 336/678] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:35:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:14.153783 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7aceb-a9bc-42b4-ba42-724a77fd77fe None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:14.158203 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 343/679] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:35:14.161361 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475a127-4f9e-4885-beaf-ba95bd49473f None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:14.187727 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:14.268186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:14.273201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7aceb-a9bc-42b4-ba42-724a77fd77fe None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:14.276395 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475a127-4f9e-4885-beaf-ba95bd49473f None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:14.316111 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b15ef-4f41-4463-a8b3-e3fa001c4d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:14.316979 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b15ef-4f41-4463-a8b3-e3fa001c4d26 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:14.317262 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b15ef-4f41-4463-a8b3-e3fa001c4d26 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:14.317560 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b15ef-4f41-4463-a8b3-e3fa001c4d26 None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:14.318198 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b15ef-4f41-4463-a8b3-e3fa001c4d26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:14.318499 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b15ef-4f41-4463-a8b3-e3fa001c4d26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:14.320204 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b15ef-4f41-4463-a8b3-e3fa001c4d26 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:14.323009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d25c9f-4a83-4f1f-be05-4b2db93f7192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:14.323009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d25c9f-4a83-4f1f-be05-4b2db93f7192 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:14.323009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d25c9f-4a83-4f1f-be05-4b2db93f7192 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:14.323009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d25c9f-4a83-4f1f-be05-4b2db93f7192 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:14.329259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d25c9f-4a83-4f1f-be05-4b2db93f7192 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:14.330953 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d25c9f-4a83-4f1f-be05-4b2db93f7192 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:14.340629 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 337/680] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:14.340629 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b15ef-4f41-4463-a8b3-e3fa001c4d26 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:14.346347 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-97d25c9f-4a83-4f1f-be05-4b2db93f7192 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 22:35:14.346347 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:14.346347 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:14.346347 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:14.346347 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:14.346347 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:14.346347 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:14.346347 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:14.346347 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:14.346347 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:14.346347 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:14.346347 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:14.346347 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:35:14.346347 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:35:14.346347 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:35:14.346347 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:35:14.347503 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:14.347503 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:14.347503 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:35:14.347503 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:14.347503 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:35:14.347503 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:14.347503 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:14.347503 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:14.347503 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:14.347503 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:14.347503 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:14.347503 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:14.347503 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:14.347503 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:14.347503 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:14.347503 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:14.347503 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:14.347503 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:14.348681 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:14.348681 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:14.348681 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:35:14.348681 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:35:14.348681 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:35:14.348681 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:35:14.348681 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:35:14.348681 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:35:14.348681 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 22:35:14.348681 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:14.354195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 344/681] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:35:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:14.354195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d25c9f-4a83-4f1f-be05-4b2db93f7192 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:14.441720 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:14.451506 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b15ef-4f41-4463-a8b3-e3fa001c4d26 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:14.475962 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d25c9f-4a83-4f1f-be05-4b2db93f7192 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:14.488349 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec9fb0-6bf5-4646-9c6c-9e4c1bb1de69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:14.489404 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec9fb0-6bf5-4646-9c6c-9e4c1bb1de69 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:14.489873 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec9fb0-6bf5-4646-9c6c-9e4c1bb1de69 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:14.490210 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec9fb0-6bf5-4646-9c6c-9e4c1bb1de69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:14.490996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ec9fb0-6bf5-4646-9c6c-9e4c1bb1de69 None admin] RBAC: Adding query filter params (role.id=5dfd0f952cd641bbb2257860c4c892ac, scope.project.id=3caa2ea846f7449b8c483788fc5846c8, user.id=dc7524edf5d5422f91d92fd91be5ee1e) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:14.491380 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ec9fb0-6bf5-4646-9c6c-9e4c1bb1de69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:14.492874 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ec9fb0-6bf5-4646-9c6c-9e4c1bb1de69 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:14.505614 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 338/682] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:35:14 2022] GET /identity/v3/role_assignments?user.id=dc7524edf5d5422f91d92fd91be5ee1e&scope.project.id=3caa2ea846f7449b8c483788fc5846c8&role.id=5dfd0f952cd641bbb2257860c4c892ac => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:14.505614 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec9fb0-6bf5-4646-9c6c-9e4c1bb1de69 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:14.528210 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cc9e17-88c1-4199-82fc-247fcba9bb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:14.528865 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc9e17-88c1-4199-82fc-247fcba9bb94 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:14.529190 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc9e17-88c1-4199-82fc-247fcba9bb94 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:14.529493 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc9e17-88c1-4199-82fc-247fcba9bb94 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:14.544113 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cc9e17-88c1-4199-82fc-247fcba9bb94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:35:14.549624 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-d9cc9e17-88c1-4199-82fc-247fcba9bb94 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:35:14.549624 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:14.549624 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:14.549624 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:14.549624 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:14.549624 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:14.549624 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:14.549624 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:14.549624 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:14.549624 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:14.549624 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:14.549624 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:14.549624 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:35:14.549624 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:35:14.549624 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:35:14.549624 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:35:14.550704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:35:14.550704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:35:14.550704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:35:14.550704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:35:14.550704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:14.550704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:14.550704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:14.550704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:14.550704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:14.550704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:14.550704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:14.550704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:14.550704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:14.550704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:14.550704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:14.550704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:14.551557 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:14.551557 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:14.551557 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:14.551557 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:14.551557 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:35:14.551557 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:35:14.551557 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:35:14.551557 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:35:14.551557 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:35:14.551557 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:35:14.551557 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:35:14.551557 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:14.555927 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 345/683] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:35:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:14.556943 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cc9e17-88c1-4199-82fc-247fcba9bb94 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:14.580304 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:14.644990 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec9fb0-6bf5-4646-9c6c-9e4c1bb1de69 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:14.680844 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cc9e17-88c1-4199-82fc-247fcba9bb94 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:14.691409 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed42b7-0f82-4e2b-a2fa-9a4876725660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:14.692772 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed42b7-0f82-4e2b-a2fa-9a4876725660 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:14.693362 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed42b7-0f82-4e2b-a2fa-9a4876725660 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:14.693820 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed42b7-0f82-4e2b-a2fa-9a4876725660 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:14.705270 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ed42b7-0f82-4e2b-a2fa-9a4876725660 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:14.707942 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ed42b7-0f82-4e2b-a2fa-9a4876725660 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:14.722283 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-64ed42b7-0f82-4e2b-a2fa-9a4876725660 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:14.722283 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:14.722283 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:14.722283 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:14.722283 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:14.722283 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:14.722283 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:14.722283 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:14.722283 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:14.722283 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:14.722283 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:14.722283 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:14.722283 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:35:14.722283 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:35:14.722283 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:35:14.723548 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:35:14.723548 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:14.723548 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:14.723548 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:14.723548 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:14.723548 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:14.723548 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:14.723548 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:14.723548 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:14.723548 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:14.723548 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:14.723548 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:14.723548 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:14.723548 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:14.723548 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:14.723548 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:14.723548 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:14.724440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:35:14.724440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:35:14.724440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:35:14.724440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:35:14.724440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:35:14.724440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:35:14.724440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:14.724440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:14.724870 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86da160-e39a-4a6e-b272-e8111e219bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:14.724870 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86da160-e39a-4a6e-b272-e8111e219bf2 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:14.724870 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86da160-e39a-4a6e-b272-e8111e219bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:14.725635 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86da160-e39a-4a6e-b272-e8111e219bf2 None admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:14.725878 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86da160-e39a-4a6e-b272-e8111e219bf2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:14.726182 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86da160-e39a-4a6e-b272-e8111e219bf2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:14.726731 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 339/684] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:35:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:14.729695 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed42b7-0f82-4e2b-a2fa-9a4876725660 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:14.729815 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86da160-e39a-4a6e-b272-e8111e219bf2 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:14.747280 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 346/685] 10.222.0.76 () {56 vars in 1135 bytes} [Wed Aug 10 22:35:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:14.748053 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86da160-e39a-4a6e-b272-e8111e219bf2 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:14.750897 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:14.837675 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:14.873330 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86da160-e39a-4a6e-b272-e8111e219bf2 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:14.874145 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed42b7-0f82-4e2b-a2fa-9a4876725660 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:14.912226 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1640eb4b-6d5e-4339-a43f-8243090e228c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:14.912844 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640eb4b-6d5e-4339-a43f-8243090e228c None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:14.913458 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640eb4b-6d5e-4339-a43f-8243090e228c None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:14.913751 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640eb4b-6d5e-4339-a43f-8243090e228c None admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:14.914739 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1640eb4b-6d5e-4339-a43f-8243090e228c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:14.915047 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1640eb4b-6d5e-4339-a43f-8243090e228c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:14.916488 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1640eb4b-6d5e-4339-a43f-8243090e228c None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:14.919571 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb5acd0-ceab-4656-9ff3-8e5b83336f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:14.920698 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5acd0-ceab-4656-9ff3-8e5b83336f5d None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:14.921209 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5acd0-ceab-4656-9ff3-8e5b83336f5d None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:14.921672 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5acd0-ceab-4656-9ff3-8e5b83336f5d None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:14.922493 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb5acd0-ceab-4656-9ff3-8e5b83336f5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:14.922950 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb5acd0-ceab-4656-9ff3-8e5b83336f5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:14.925405 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb5acd0-ceab-4656-9ff3-8e5b83336f5d None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:14.930330 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 347/686] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:35:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:14.931184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1640eb4b-6d5e-4339-a43f-8243090e228c None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:14.941619 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 340/687] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:35:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:14.942851 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb5acd0-ceab-4656-9ff3-8e5b83336f5d None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:15.043694 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:15.069983 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1640eb4b-6d5e-4339-a43f-8243090e228c None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:15.077001 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb5acd0-ceab-4656-9ff3-8e5b83336f5d None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:15.111780 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:15.124306 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d701c-c92b-4f88-87c2-d69f97897760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:15.125311 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d701c-c92b-4f88-87c2-d69f97897760 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:15.125735 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d701c-c92b-4f88-87c2-d69f97897760 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:15.126151 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d701c-c92b-4f88-87c2-d69f97897760 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:15.128532 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb5091-55b1-40d8-95e9-ec28dc500aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:15.129371 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb5091-55b1-40d8-95e9-ec28dc500aa8 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:15.129817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb5091-55b1-40d8-95e9-ec28dc500aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:15.130526 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb5091-55b1-40d8-95e9-ec28dc500aa8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:15.132456 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb5091-55b1-40d8-95e9-ec28dc500aa8 None admin] RBAC: Adding query filter params (role.id=5dfd0f952cd641bbb2257860c4c892ac, scope.project.id=3caa2ea846f7449b8c483788fc5846c8, user.id=7fcb5a674ff94b3d8f79efbbe4959069) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:15.132937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb5091-55b1-40d8-95e9-ec28dc500aa8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:15.134572 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb5091-55b1-40d8-95e9-ec28dc500aa8 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:15.143091 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 341/688] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:35:14 2022] GET /identity/v3/role_assignments?user.id=7fcb5a674ff94b3d8f79efbbe4959069&scope.project.id=3caa2ea846f7449b8c483788fc5846c8&role.id=5dfd0f952cd641bbb2257860c4c892ac => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:15.144870 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb5091-55b1-40d8-95e9-ec28dc500aa8 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:15.249871 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324d701c-c92b-4f88-87c2-d69f97897760 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:15.251305 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324d701c-c92b-4f88-87c2-d69f97897760 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:15.336173 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb5091-55b1-40d8-95e9-ec28dc500aa8 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:15.379092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2d38e-b705-458b-9e69-63e98abceddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:15.379792 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2d38e-b705-458b-9e69-63e98abceddd None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:15.380068 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2d38e-b705-458b-9e69-63e98abceddd None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:15.380335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2d38e-b705-458b-9e69-63e98abceddd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:15.394133 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:15.397750 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc2d38e-b705-458b-9e69-63e98abceddd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:15.399769 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc2d38e-b705-458b-9e69-63e98abceddd None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:15.423223 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-9bc2d38e-b705-458b-9e69-63e98abceddd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:15.423223 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:15.423223 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:15.423223 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:15.423223 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:15.423223 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:15.423223 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:15.423223 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:15.423223 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:15.423223 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:15.423223 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:15.423223 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:15.423223 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:15.423223 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:15.423223 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:15.423223 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:15.424230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:15.424230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:15.424230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:15.424230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:15.424230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:15.424230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:15.424230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:15.424230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:15.424230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:15.424230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:15.424230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:15.424230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:15.424230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:15.424230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:15.424230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:15.424230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:15.424230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:15.425779 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:15.425779 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:15.425779 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:15.429788 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 342/689] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:15.432315 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2d38e-b705-458b-9e69-63e98abceddd None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:15.465601 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 348/690] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:35:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:35:15.471337 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d701c-c92b-4f88-87c2-d69f97897760 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:15.505927 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:15.554546 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2d38e-b705-458b-9e69-63e98abceddd None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:15.583739 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d701c-c92b-4f88-87c2-d69f97897760 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:15.600302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5fedc-cb63-4c14-81e9-58619853a24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:15.601132 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5fedc-cb63-4c14-81e9-58619853a24e None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:15.601636 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5fedc-cb63-4c14-81e9-58619853a24e None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:15.601994 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5fedc-cb63-4c14-81e9-58619853a24e None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:15.602705 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea5fedc-cb63-4c14-81e9-58619853a24e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:15.603072 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea5fedc-cb63-4c14-81e9-58619853a24e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:15.604954 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea5fedc-cb63-4c14-81e9-58619853a24e None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:15.621071 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 343/691] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:15.626896 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee357d8-d624-4c39-8a62-ca149bc8b646 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:15.627231 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee357d8-d624-4c39-8a62-ca149bc8b646 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:15.627487 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee357d8-d624-4c39-8a62-ca149bc8b646 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:15.628751 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc36483d-8ccc-4b70-9668-971f08824e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:15.629626 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36483d-8ccc-4b70-9668-971f08824e94 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:15.629833 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 344/692] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:15.630275 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36483d-8ccc-4b70-9668-971f08824e94 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:15.630646 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee357d8-d624-4c39-8a62-ca149bc8b646 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:15.630999 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36483d-8ccc-4b70-9668-971f08824e94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:15.642967 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc36483d-8ccc-4b70-9668-971f08824e94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:15.645332 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc36483d-8ccc-4b70-9668-971f08824e94 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:15.661009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-bc36483d-8ccc-4b70-9668-971f08824e94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:15.661009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:15.661009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:15.661009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:15.661009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:15.661009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:15.661009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:15.661009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:15.661009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:15.661009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:15.661009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:15.661009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:15.661009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:15.661009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:15.661009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:15.661009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:15.662311 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:15.662311 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:15.662311 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:15.662311 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:15.662311 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:15.662311 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:15.662311 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:15.662311 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:15.662311 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:15.662311 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:15.662311 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:15.662311 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:15.662311 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:15.662311 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:15.662311 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:15.662311 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:15.662311 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:15.663229 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:15.663229 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:15.663229 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:15.665304 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 349/693] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:15.666733 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f5060-3f7b-435e-b199-8f102cf7c360 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:15.667199 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f5060-3f7b-435e-b199-8f102cf7c360 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:15.667680 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f5060-3f7b-435e-b199-8f102cf7c360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:15.687023 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-239f5060-3f7b-435e-b199-8f102cf7c360 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:15.688591 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-239f5060-3f7b-435e-b199-8f102cf7c360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:15.694442 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:15.757429 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee357d8-d624-4c39-8a62-ca149bc8b646 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:15.764884 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 350/694] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:35:15.766383 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239f5060-3f7b-435e-b199-8f102cf7c360 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:15.782829 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:15.807423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae7215-a1b2-467e-947b-5ec6aba04afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:15.808078 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae7215-a1b2-467e-947b-5ec6aba04afd None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:15.808356 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae7215-a1b2-467e-947b-5ec6aba04afd None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:15.808626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae7215-a1b2-467e-947b-5ec6aba04afd None admin] PATH_INFO: `/v3/users/nova` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:15.818280 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ae7215-a1b2-467e-947b-5ec6aba04afd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:15.820285 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ae7215-a1b2-467e-947b-5ec6aba04afd None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:15.837037 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-b2ae7215-a1b2-467e-947b-5ec6aba04afd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:35:15.837037 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:15.837037 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:15.837037 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:15.837037 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:15.837037 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:15.837037 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:15.837037 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:15.837037 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:15.837037 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:15.837037 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:15.837037 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:15.837037 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:35:15.837037 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:35:15.837037 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:35:15.837037 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:35:15.838110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:15.838110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:15.838110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:35:15.838110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:15.838110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:35:15.838110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:15.838110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:15.838110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:15.838110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:15.838110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:15.838110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:15.838110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:15.838110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:15.838110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:15.838110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:15.838110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:15.838110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:15.838110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:15.839034 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:15.839034 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:15.839034 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:35:15.839034 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:35:15.839034 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:35:15.839034 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:35:15.839034 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:35:15.839034 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:35:15.839034 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:35:15.839034 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:15.840477 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 345/695] 10.222.0.76 () {56 vars in 1093 bytes} [Wed Aug 10 22:35:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:15.845438 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae7215-a1b2-467e-947b-5ec6aba04afd None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:15.907264 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239f5060-3f7b-435e-b199-8f102cf7c360 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:15.950539 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b67f5e-75cb-4618-9447-1085e0f0b728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:15.951603 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b67f5e-75cb-4618-9447-1085e0f0b728 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:15.952090 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b67f5e-75cb-4618-9447-1085e0f0b728 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:15.952586 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b67f5e-75cb-4618-9447-1085e0f0b728 None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:15.953466 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b67f5e-75cb-4618-9447-1085e0f0b728 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:15.953994 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b67f5e-75cb-4618-9447-1085e0f0b728 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:15.955728 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b67f5e-75cb-4618-9447-1085e0f0b728 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:15.970780 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 351/696] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:15.971811 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b67f5e-75cb-4618-9447-1085e0f0b728 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:15.978381 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae7215-a1b2-467e-947b-5ec6aba04afd None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:15.997348 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:16.027140 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01074a3-73ff-4b3e-9dcf-5df71885e55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:16.028268 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01074a3-73ff-4b3e-9dcf-5df71885e55c None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:16.028723 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01074a3-73ff-4b3e-9dcf-5df71885e55c None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:16.029149 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01074a3-73ff-4b3e-9dcf-5df71885e55c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:16.029905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01074a3-73ff-4b3e-9dcf-5df71885e55c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:16.031872 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01074a3-73ff-4b3e-9dcf-5df71885e55c None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:16.045871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-f01074a3-73ff-4b3e-9dcf-5df71885e55c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 22:35:16.045871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:16.045871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:35:16.045871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:35:16.045871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:35:16.045871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:35:16.045871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:35:16.045871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:16.045871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 22:35:16.045871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:16.045871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:35:16.045871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:16.045871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:16.045871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:16.045871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:16.045871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:16.045871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:16.047046 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:16.047046 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:16.047046 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:16.047046 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:16.047046 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:16.047046 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:16.047046 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:35:16.047046 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:35:16.047046 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:35:16.047046 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:35:16.047046 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:16.047046 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:16.047046 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:16.047046 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:16.047046 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:16.047046 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:16.047921 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:16.047921 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:16.047921 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:16.047921 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:16.047921 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:16.047921 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:16.047921 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:16.047921 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:16.047921 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:16.047921 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:16.047921 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:35:16.047921 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:16.047921 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 22:35:16.047921 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:16.049089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 346/697] 10.222.0.76 () {56 vars in 1129 bytes} [Wed Aug 10 22:35:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:16.050571 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01074a3-73ff-4b3e-9dcf-5df71885e55c None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:16.073934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:16.098365 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b67f5e-75cb-4618-9447-1085e0f0b728 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:16.143574 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547146b-cfc3-4ef0-87ca-1fb58786aee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:16.144897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547146b-cfc3-4ef0-87ca-1fb58786aee8 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:16.145519 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547146b-cfc3-4ef0-87ca-1fb58786aee8 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:16.145948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547146b-cfc3-4ef0-87ca-1fb58786aee8 None admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:16.147136 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5547146b-cfc3-4ef0-87ca-1fb58786aee8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:16.147594 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5547146b-cfc3-4ef0-87ca-1fb58786aee8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:16.149190 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5547146b-cfc3-4ef0-87ca-1fb58786aee8 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:16.167056 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01074a3-73ff-4b3e-9dcf-5df71885e55c None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:16.173569 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 352/698] 10.222.0.76 () {56 vars in 1133 bytes} [Wed Aug 10 22:35:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:16.175298 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547146b-cfc3-4ef0-87ca-1fb58786aee8 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:16.212930 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf2316e-81a5-4e67-a89d-d280da45b176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:16.213612 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2316e-81a5-4e67-a89d-d280da45b176 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:16.214072 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2316e-81a5-4e67-a89d-d280da45b176 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:16.214469 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2316e-81a5-4e67-a89d-d280da45b176 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:16.223022 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf2316e-81a5-4e67-a89d-d280da45b176 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:16.224867 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf2316e-81a5-4e67-a89d-d280da45b176 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:16.238929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-abf2316e-81a5-4e67-a89d-d280da45b176 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:16.238929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:16.238929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:16.238929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:16.238929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:16.238929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:16.238929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:16.238929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:16.238929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:16.238929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:16.238929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:16.238929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:16.238929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:35:16.238929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:35:16.238929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:35:16.240090 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:35:16.240090 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:16.240090 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:16.240090 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:16.240090 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:16.240090 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:16.240090 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:16.240090 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:16.240090 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:16.240090 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:16.240090 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:16.240090 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:16.240090 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:16.240090 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:16.240090 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:16.240090 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:16.240090 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:16.241188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:35:16.241188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:35:16.241188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:35:16.241188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:35:16.241188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:35:16.241188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:35:16.241188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:16.241188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:16.242544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 347/699] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:35:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:16.243478 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf2316e-81a5-4e67-a89d-d280da45b176 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:16.285060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:16.323316 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547146b-cfc3-4ef0-87ca-1fb58786aee8 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:16.358502 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:16.367066 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d249fca-1c71-439a-9771-dffdb40d6d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:16.367690 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d249fca-1c71-439a-9771-dffdb40d6d64 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:16.367973 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d249fca-1c71-439a-9771-dffdb40d6d64 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:16.368247 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d249fca-1c71-439a-9771-dffdb40d6d64 None admin] PATH_INFO: `/v3/services` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:16.368888 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d249fca-1c71-439a-9771-dffdb40d6d64 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:16.369336 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d249fca-1c71-439a-9771-dffdb40d6d64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:16.369711 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf2316e-81a5-4e67-a89d-d280da45b176 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:16.370722 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d249fca-1c71-439a-9771-dffdb40d6d64 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:16.381673 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 353/700] 10.222.0.76 () {56 vars in 1124 bytes} [Wed Aug 10 22:35:16 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:16.383006 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d249fca-1c71-439a-9771-dffdb40d6d64 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:16.413544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a81a21-01b8-4100-a896-a662e7574bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:16.418019 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a81a21-01b8-4100-a896-a662e7574bf9 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:16.418301 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a81a21-01b8-4100-a896-a662e7574bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:16.419661 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a81a21-01b8-4100-a896-a662e7574bf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:16.522974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a81a21-01b8-4100-a896-a662e7574bf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:16.524440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a81a21-01b8-4100-a896-a662e7574bf9 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:16.525681 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d249fca-1c71-439a-9771-dffdb40d6d64 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:16.564583 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:16.578104 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1bb8ef-ee9f-4052-9795-0b4c824ff0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:16.579085 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1bb8ef-ee9f-4052-9795-0b4c824ff0f8 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:16.579598 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1bb8ef-ee9f-4052-9795-0b4c824ff0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:16.580076 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1bb8ef-ee9f-4052-9795-0b4c824ff0f8 None admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:16.581560 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1bb8ef-ee9f-4052-9795-0b4c824ff0f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:16.582128 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1bb8ef-ee9f-4052-9795-0b4c824ff0f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:16.583780 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1bb8ef-ee9f-4052-9795-0b4c824ff0f8 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:16.596981 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 354/701] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:35:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:16.598187 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1bb8ef-ee9f-4052-9795-0b4c824ff0f8 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:16.641334 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 348/702] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:35:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:35:16.642273 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a81a21-01b8-4100-a896-a662e7574bf9 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:16.662737 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:16.712269 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1bb8ef-ee9f-4052-9795-0b4c824ff0f8 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:16.748351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a81a21-01b8-4100-a896-a662e7574bf9 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:16.750889 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c313b1-b6af-473a-8ad7-133213205197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:16.751766 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c313b1-b6af-473a-8ad7-133213205197 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:16.752286 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c313b1-b6af-473a-8ad7-133213205197 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:16.753001 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c313b1-b6af-473a-8ad7-133213205197 None admin] PATH_INFO: `/v3/services` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:16.753914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c313b1-b6af-473a-8ad7-133213205197 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:16.754427 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c313b1-b6af-473a-8ad7-133213205197 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:16.756017 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c313b1-b6af-473a-8ad7-133213205197 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:16.762598 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 355/703] 10.222.0.76 () {56 vars in 1124 bytes} [Wed Aug 10 22:35:16 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:16.763760 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c313b1-b6af-473a-8ad7-133213205197 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:16.786533 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37b04c-3485-4ac2-b79b-8edab1e532e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:16.787209 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37b04c-3485-4ac2-b79b-8edab1e532e1 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:16.787511 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37b04c-3485-4ac2-b79b-8edab1e532e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:16.788025 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37b04c-3485-4ac2-b79b-8edab1e532e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:16.788609 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be37b04c-3485-4ac2-b79b-8edab1e532e1 None admin] RBAC: Adding query filter params (role.id=5dfd0f952cd641bbb2257860c4c892ac, scope.project.id=3caa2ea846f7449b8c483788fc5846c8, user.id=1e3a76df95a34b9fb9816f22b85d1071) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:16.788918 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be37b04c-3485-4ac2-b79b-8edab1e532e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:16.790512 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be37b04c-3485-4ac2-b79b-8edab1e532e1 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:16.793991 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:16.798660 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 349/704] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:35:16 2022] GET /identity/v3/role_assignments?user.id=1e3a76df95a34b9fb9816f22b85d1071&scope.project.id=3caa2ea846f7449b8c483788fc5846c8&role.id=5dfd0f952cd641bbb2257860c4c892ac => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:16.812920 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecdba9f-5a2c-4088-9fce-cbad3254f4ec None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:16.813265 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecdba9f-5a2c-4088-9fce-cbad3254f4ec None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:16.817770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecdba9f-5a2c-4088-9fce-cbad3254f4ec None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:16.819488 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 350/705] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:16.825055 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809c62d-281c-4c0d-935b-735aaa859503 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:16.825404 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809c62d-281c-4c0d-935b-735aaa859503 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:16.825640 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809c62d-281c-4c0d-935b-735aaa859503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:16.872830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-8809c62d-281c-4c0d-935b-735aaa859503 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:16.879882 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-8809c62d-281c-4c0d-935b-735aaa859503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:16.957910 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c313b1-b6af-473a-8ad7-133213205197 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:16.983697 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:16.995303 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 351/706] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:35:17.007007 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc98658-2278-4da5-9bae-cb0c21a220a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:17.007658 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc98658-2278-4da5-9bae-cb0c21a220a0 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:17.007941 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc98658-2278-4da5-9bae-cb0c21a220a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:17.008220 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc98658-2278-4da5-9bae-cb0c21a220a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:17.024501 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8809c62d-281c-4c0d-935b-735aaa859503 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:17.144260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc98658-2278-4da5-9bae-cb0c21a220a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:17.146131 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc98658-2278-4da5-9bae-cb0c21a220a0 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:17.159250 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8809c62d-281c-4c0d-935b-735aaa859503 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:17.162020 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-bfc98658-2278-4da5-9bae-cb0c21a220a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:17.162020 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:17.162020 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:17.162020 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:17.162020 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:17.162020 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:17.162020 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:17.162020 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:17.162020 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:17.162020 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:17.162020 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:17.162020 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:17.162020 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:17.162020 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:17.162020 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:17.162020 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:17.162953 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:17.162953 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:17.162953 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:17.162953 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:17.162953 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:17.162953 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:17.162953 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:17.162953 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:17.162953 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:17.162953 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:17.162953 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:17.162953 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:17.162953 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:17.162953 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:17.162953 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:17.162953 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:17.162953 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:17.164009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:17.164009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:17.164009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:17.164009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 356/707] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:17.168488 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc98658-2278-4da5-9bae-cb0c21a220a0 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:17.196218 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:17.203973 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554310e1-f518-499d-a46a-17f2f71c7817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:17.206241 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554310e1-f518-499d-a46a-17f2f71c7817 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:17.206241 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554310e1-f518-499d-a46a-17f2f71c7817 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:17.206525 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554310e1-f518-499d-a46a-17f2f71c7817 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:17.220760 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554310e1-f518-499d-a46a-17f2f71c7817 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:17.223090 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554310e1-f518-499d-a46a-17f2f71c7817 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:17.242205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-554310e1-f518-499d-a46a-17f2f71c7817 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:17.242205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:17.242205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:17.242205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:17.242205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:17.242205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:17.242205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:17.242205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:17.242205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:17.242205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:17.242205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:17.242205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:17.242205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:17.242205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:17.242205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:17.242205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:17.243292 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:17.243292 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:17.243292 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:17.243292 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:17.243292 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:17.243292 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:17.243292 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:17.243292 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:17.243292 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:17.243292 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:17.243292 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:17.243292 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:17.243292 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:17.243292 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:17.243292 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:17.243292 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:17.243292 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:17.244188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:17.244188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:17.244188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:17.246333 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 352/708] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:35:17.253063 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554310e1-f518-499d-a46a-17f2f71c7817 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:17.311883 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc98658-2278-4da5-9bae-cb0c21a220a0 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:17.367439 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace05e0-ca2c-4f6f-b0f2-69c9a332a64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:17.368385 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace05e0-ca2c-4f6f-b0f2-69c9a332a64a None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:17.368850 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace05e0-ca2c-4f6f-b0f2-69c9a332a64a None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:17.369321 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace05e0-ca2c-4f6f-b0f2-69c9a332a64a None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:17.370740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace05e0-ca2c-4f6f-b0f2-69c9a332a64a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:17.371226 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace05e0-ca2c-4f6f-b0f2-69c9a332a64a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:17.373753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace05e0-ca2c-4f6f-b0f2-69c9a332a64a None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:17.391179 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 357/709] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:17.396613 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace05e0-ca2c-4f6f-b0f2-69c9a332a64a None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:17.404677 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554310e1-f518-499d-a46a-17f2f71c7817 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:17.439001 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:17.495393 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4192db-8a6c-412b-bb81-39e579c63343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:17.496286 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4192db-8a6c-412b-bb81-39e579c63343 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:17.496479 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4192db-8a6c-412b-bb81-39e579c63343 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:17.496882 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4192db-8a6c-412b-bb81-39e579c63343 None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:17.497692 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4192db-8a6c-412b-bb81-39e579c63343 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:17.498069 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4192db-8a6c-412b-bb81-39e579c63343 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:17.499532 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4192db-8a6c-412b-bb81-39e579c63343 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:17.521329 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 353/710] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:35:17.526865 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4192db-8a6c-412b-bb81-39e579c63343 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:17.527963 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace05e0-ca2c-4f6f-b0f2-69c9a332a64a None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:17.572038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7439de2a-1abe-4620-a97f-ecada0d29091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:17.573361 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439de2a-1abe-4620-a97f-ecada0d29091 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:17.574133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439de2a-1abe-4620-a97f-ecada0d29091 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:17.574616 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439de2a-1abe-4620-a97f-ecada0d29091 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:17.584979 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7439de2a-1abe-4620-a97f-ecada0d29091 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:17.587693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7439de2a-1abe-4620-a97f-ecada0d29091 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:17.600878 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:17.606232 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-7439de2a-1abe-4620-a97f-ecada0d29091 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:17.606232 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:17.606232 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:17.606232 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:17.606232 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:17.606232 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:17.606232 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:17.606232 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:17.606232 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:17.606232 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:17.606232 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:17.606232 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:17.606232 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:35:17.606232 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:35:17.606232 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:35:17.607146 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:35:17.607146 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:17.607146 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:17.607146 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:17.607146 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:17.607146 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:17.607146 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:17.607146 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:17.607146 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:17.607146 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:17.607146 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:17.607146 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:17.607146 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:17.607146 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:17.607146 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:17.607146 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:17.607146 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:17.608100 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:35:17.608100 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:35:17.608100 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:35:17.608100 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:35:17.608100 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:35:17.608100 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:35:17.608100 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:17.608100 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:17.610633 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 358/711] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:35:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:35:17.612038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d385f-e202-4c24-bd4f-4415ad6891ec None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:17.612723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d385f-e202-4c24-bd4f-4415ad6891ec None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:17.613194 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d385f-e202-4c24-bd4f-4415ad6891ec None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:17.614846 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 359/712] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:17.619467 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d385f-e202-4c24-bd4f-4415ad6891ec None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:17.677094 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4192db-8a6c-412b-bb81-39e579c63343 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:17.715781 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c9bba-9002-4308-a83c-4733b3608505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:17.716451 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c9bba-9002-4308-a83c-4733b3608505 None admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:17.716790 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c9bba-9002-4308-a83c-4733b3608505 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:17.717072 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c9bba-9002-4308-a83c-4733b3608505 None admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:17.717797 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40c9bba-9002-4308-a83c-4733b3608505 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:17.719313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40c9bba-9002-4308-a83c-4733b3608505 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:17.743340 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:17.753309 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 354/713] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 10 22:35:17 2022] POST /identity/v3/users => generated 254 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:35:17.753309 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d5e94-d8aa-4c3b-9dc9-47d4df4d012b None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:17.753309 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d5e94-d8aa-4c3b-9dc9-47d4df4d012b None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:17.753309 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d5e94-d8aa-4c3b-9dc9-47d4df4d012b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:17.771801 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d385f-e202-4c24-bd4f-4415ad6891ec None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:17.790998 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-385d5e94-d8aa-4c3b-9dc9-47d4df4d012b None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:17.792237 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-385d5e94-d8aa-4c3b-9dc9-47d4df4d012b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:17.845316 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bfe3bd-5fb7-47e1-bd7c-a1857ab9e046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:17.845950 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bfe3bd-5fb7-47e1-bd7c-a1857ab9e046 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:17.846906 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bfe3bd-5fb7-47e1-bd7c-a1857ab9e046 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:17.847248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bfe3bd-5fb7-47e1-bd7c-a1857ab9e046 None admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:17.847961 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bfe3bd-5fb7-47e1-bd7c-a1857ab9e046 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:17.848268 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bfe3bd-5fb7-47e1-bd7c-a1857ab9e046 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:17.849775 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bfe3bd-5fb7-47e1-bd7c-a1857ab9e046 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:17.884388 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 360/714] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:35:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:35:17.890722 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bfe3bd-5fb7-47e1-bd7c-a1857ab9e046 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:17.914528 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 355/715] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:35:17.942421 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d5e94-d8aa-4c3b-9dc9-47d4df4d012b None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:17.983139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:18.060958 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bfe3bd-5fb7-47e1-bd7c-a1857ab9e046 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:18.085724 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d5e94-d8aa-4c3b-9dc9-47d4df4d012b None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:18.111418 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0586f9b-aadb-462c-a4c0-01c17c55c1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:18.112982 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0586f9b-aadb-462c-a4c0-01c17c55c1e4 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:18.113514 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0586f9b-aadb-462c-a4c0-01c17c55c1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:18.113998 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0586f9b-aadb-462c-a4c0-01c17c55c1e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:18.114909 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0586f9b-aadb-462c-a4c0-01c17c55c1e4 None admin] RBAC: Adding query filter params (role.id=6641e95fdc1f4c098ebc372bb692d227, scope.project.id=3caa2ea846f7449b8c483788fc5846c8, user.id=b22838a8ad3949e59cdbaaa1801cd07f) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:18.115409 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0586f9b-aadb-462c-a4c0-01c17c55c1e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:18.117109 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0586f9b-aadb-462c-a4c0-01c17c55c1e4 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:18.123709 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 361/716] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:35:17 2022] GET /identity/v3/role_assignments?user.id=b22838a8ad3949e59cdbaaa1801cd07f&scope.project.id=3caa2ea846f7449b8c483788fc5846c8&role.id=6641e95fdc1f4c098ebc372bb692d227 => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:35:18.128452 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86978245-a155-4949-bbfa-2290d6948388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:18.129503 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86978245-a155-4949-bbfa-2290d6948388 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:18.129994 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86978245-a155-4949-bbfa-2290d6948388 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:18.130486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86978245-a155-4949-bbfa-2290d6948388 None admin] PATH_INFO: `/v3/services/network` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:18.131292 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86978245-a155-4949-bbfa-2290d6948388 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:18.133051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86978245-a155-4949-bbfa-2290d6948388 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:18.144394 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-86978245-a155-4949-bbfa-2290d6948388 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 22:35:18.144394 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:18.144394 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:35:18.144394 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:35:18.144394 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:35:18.144394 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:35:18.144394 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:35:18.144394 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:18.144394 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 22:35:18.144394 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:18.144394 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:35:18.144394 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:18.144394 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:18.144394 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:18.144394 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:18.144394 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:18.144394 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:18.145544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:18.145544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:18.145544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:18.145544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:18.145544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:18.145544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:18.145544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:35:18.145544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:35:18.145544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:35:18.145544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:35:18.145544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:18.145544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:18.145544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:18.145544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:18.145544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:18.145544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:18.146406 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:18.146406 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:18.146406 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:18.146406 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:18.146406 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:18.146406 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:18.146406 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:18.146406 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:18.146406 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:18.146406 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:18.146406 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:35:18.146406 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:18.146406 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 22:35:18.146406 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:18.149477 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 356/717] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:35:17 2022] GET /identity/v3/services/network => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:35:18.155789 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0586f9b-aadb-462c-a4c0-01c17c55c1e4 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:18.179298 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:18.274834 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0586f9b-aadb-462c-a4c0-01c17c55c1e4 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:18.313070 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6633dfc2-4f5b-4059-916c-c38db920126f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:18.314102 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633dfc2-4f5b-4059-916c-c38db920126f None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:18.314620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633dfc2-4f5b-4059-916c-c38db920126f None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:18.315681 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633dfc2-4f5b-4059-916c-c38db920126f None admin] PATH_INFO: `/v3/services` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:18.316565 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6633dfc2-4f5b-4059-916c-c38db920126f None admin] RBAC: Adding query filter params (name=network) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:18.317159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6633dfc2-4f5b-4059-916c-c38db920126f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:18.318784 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6633dfc2-4f5b-4059-916c-c38db920126f None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:18.324236 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 362/718] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 10 22:35:18 2022] GET /identity/v3/services?name=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:18.330113 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86978245-a155-4949-bbfa-2290d6948388 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:18.452354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:18.468993 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86978245-a155-4949-bbfa-2290d6948388 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:18.516719 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e691c2-aa57-460f-a9d3-dc06b6832587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:18.518326 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e691c2-aa57-460f-a9d3-dc06b6832587 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:18.519052 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e691c2-aa57-460f-a9d3-dc06b6832587 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:18.519537 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e691c2-aa57-460f-a9d3-dc06b6832587 None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:18.520366 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e691c2-aa57-460f-a9d3-dc06b6832587 None admin] RBAC: Adding query filter params (type=network) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:18.520866 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e691c2-aa57-460f-a9d3-dc06b6832587 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:18.526620 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e691c2-aa57-460f-a9d3-dc06b6832587 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:18.533197 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 357/719] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 10 22:35:18 2022] GET /identity/v3/services?type=network => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:19.251810 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27540d7e-21d3-47c6-8f5c-3ae70b64ed66 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:19.253474 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27540d7e-21d3-47c6-8f5c-3ae70b64ed66 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:19.254061 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27540d7e-21d3-47c6-8f5c-3ae70b64ed66 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:19.255813 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 363/720] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:19.261406 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc763f15-eda0-4299-b06e-41bdbdf192d8 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:19.262243 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc763f15-eda0-4299-b06e-41bdbdf192d8 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:19.262739 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc763f15-eda0-4299-b06e-41bdbdf192d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:19.285644 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-dc763f15-eda0-4299-b06e-41bdbdf192d8 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:19.289164 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-dc763f15-eda0-4299-b06e-41bdbdf192d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:19.390522 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 358/721] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:35:19.429698 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27540d7e-21d3-47c6-8f5c-3ae70b64ed66 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:19.552958 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:19.582242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27540d7e-21d3-47c6-8f5c-3ae70b64ed66 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:19.629164 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeae083-b3b7-4301-8868-3ac6061e90cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:19.629811 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeae083-b3b7-4301-8868-3ac6061e90cc None admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:19.630085 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeae083-b3b7-4301-8868-3ac6061e90cc None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:19.630350 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeae083-b3b7-4301-8868-3ac6061e90cc None admin] PATH_INFO: `/v3/services` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:19.631001 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeae083-b3b7-4301-8868-3ac6061e90cc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:19.632371 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeae083-b3b7-4301-8868-3ac6061e90cc None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:19.652379 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 364/722] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 10 22:35:19 2022] POST /identity/v3/services => generated 251 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:35:19.693002 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29273b-feb5-457f-8127-f3861e8b5ddb None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:19.693734 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29273b-feb5-457f-8127-f3861e8b5ddb None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:19.694225 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29273b-feb5-457f-8127-f3861e8b5ddb None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:19.696063 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 359/723] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:19.702182 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e67001-99a4-4abd-88dc-476647439afd None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:19.702779 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e67001-99a4-4abd-88dc-476647439afd None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:19.703284 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e67001-99a4-4abd-88dc-476647439afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:19.725160 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-f4e67001-99a4-4abd-88dc-476647439afd None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:19.726878 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-f4e67001-99a4-4abd-88dc-476647439afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:19.833705 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 365/724] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:19 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:35:19.858442 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d29273b-feb5-457f-8127-f3861e8b5ddb None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:19.966056 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:19.973588 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d29273b-feb5-457f-8127-f3861e8b5ddb None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:20.012236 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b7759-f487-4c86-966d-8babdc39aa08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:20.012878 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b7759-f487-4c86-966d-8babdc39aa08 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:20.013159 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b7759-f487-4c86-966d-8babdc39aa08 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:20.013482 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b7759-f487-4c86-966d-8babdc39aa08 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:20.025139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b7759-f487-4c86-966d-8babdc39aa08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:35:20.030126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-860b7759-f487-4c86-966d-8babdc39aa08 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:35:20.030126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:20.030126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:20.030126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:20.030126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:20.030126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:20.030126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:20.030126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:20.030126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:20.030126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:20.030126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:20.030126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:20.030126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:35:20.030126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:35:20.030126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:35:20.030126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:35:20.031048 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:35:20.031048 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:35:20.031048 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:35:20.031048 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:35:20.031048 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:20.031048 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:20.031048 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:20.031048 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:20.031048 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:20.031048 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:20.031048 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:20.031048 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:20.031048 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:20.031048 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:20.031048 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:20.031048 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:20.031881 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:20.031881 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:20.031881 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:20.031881 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:20.031881 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:35:20.031881 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:35:20.031881 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:35:20.031881 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:35:20.031881 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:35:20.031881 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:35:20.031881 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:35:20.031881 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:20.036602 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 360/725] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:35:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:20.038602 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e67001-99a4-4abd-88dc-476647439afd None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:20.117260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:20.159605 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e67001-99a4-4abd-88dc-476647439afd None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:20.198166 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025dd41-3c35-4170-a2a4-6f99ccff2294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:20.198796 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025dd41-3c35-4170-a2a4-6f99ccff2294 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:20.199078 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025dd41-3c35-4170-a2a4-6f99ccff2294 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:20.199354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025dd41-3c35-4170-a2a4-6f99ccff2294 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:20.200026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d025dd41-3c35-4170-a2a4-6f99ccff2294 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:20.200329 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d025dd41-3c35-4170-a2a4-6f99ccff2294 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:20.201976 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d025dd41-3c35-4170-a2a4-6f99ccff2294 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:20.216250 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 366/726] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:35:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:20.218423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b7759-f487-4c86-966d-8babdc39aa08 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:20.340235 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b7759-f487-4c86-966d-8babdc39aa08 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:20.377075 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7369eb79-048d-4e8d-b2cf-87178ebc4673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:20.377682 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369eb79-048d-4e8d-b2cf-87178ebc4673 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:20.377961 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369eb79-048d-4e8d-b2cf-87178ebc4673 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:20.378234 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369eb79-048d-4e8d-b2cf-87178ebc4673 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:20.388710 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7369eb79-048d-4e8d-b2cf-87178ebc4673 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:20.390221 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7369eb79-048d-4e8d-b2cf-87178ebc4673 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:20.405905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-7369eb79-048d-4e8d-b2cf-87178ebc4673 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:20.405905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:20.405905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:20.405905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:20.405905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:20.405905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:20.405905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:20.405905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:20.405905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:20.405905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:20.405905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:20.405905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:20.405905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:20.405905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:20.405905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:20.405905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:20.406899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:20.406899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:20.406899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:20.406899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:20.406899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:20.406899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:20.406899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:20.406899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:20.406899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:20.406899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:20.406899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:20.406899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:20.406899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:20.406899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:20.406899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:20.406899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:20.406899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:20.407780 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:20.407780 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:20.407780 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:20.409935 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:20.412488 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 361/727] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:20.413874 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025dd41-3c35-4170-a2a4-6f99ccff2294 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:20.497514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa377041-b176-45de-8884-469943c57aa1 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:20.497853 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa377041-b176-45de-8884-469943c57aa1 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:20.498014 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa377041-b176-45de-8884-469943c57aa1 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:20.501377 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 362/728] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:20.518531 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68db6b5-5373-4c06-ae7c-bcc542d06172 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:20.518839 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68db6b5-5373-4c06-ae7c-bcc542d06172 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:20.519086 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68db6b5-5373-4c06-ae7c-bcc542d06172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:20.554382 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-a68db6b5-5373-4c06-ae7c-bcc542d06172 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:20.555612 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-a68db6b5-5373-4c06-ae7c-bcc542d06172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:20.563268 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025dd41-3c35-4170-a2a4-6f99ccff2294 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:20.600774 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:20.610125 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199c730-53a4-429d-8963-9cade7fe15fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:20.610125 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199c730-53a4-429d-8963-9cade7fe15fc None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:20.611944 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199c730-53a4-429d-8963-9cade7fe15fc None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:20.612057 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199c730-53a4-429d-8963-9cade7fe15fc None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:20.613147 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5199c730-53a4-429d-8963-9cade7fe15fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:20.613672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5199c730-53a4-429d-8963-9cade7fe15fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:20.622727 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5199c730-53a4-429d-8963-9cade7fe15fc None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:20.648823 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 367/729] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:20.657775 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199c730-53a4-429d-8963-9cade7fe15fc None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:20.661641 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 363/730] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:20 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 22:35:20.688186 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68db6b5-5373-4c06-ae7c-bcc542d06172 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:20.808769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199c730-53a4-429d-8963-9cade7fe15fc None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:20.831864 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68db6b5-5373-4c06-ae7c-bcc542d06172 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:20.852356 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3a5a44-ff06-4193-b308-c30effb91714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:20.853893 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a5a44-ff06-4193-b308-c30effb91714 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:20.854388 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a5a44-ff06-4193-b308-c30effb91714 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:20.854864 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a5a44-ff06-4193-b308-c30effb91714 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:20.864411 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:20.868804 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3a5a44-ff06-4193-b308-c30effb91714 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:20.870807 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3a5a44-ff06-4193-b308-c30effb91714 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:20.873468 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0691cc47-e440-42e4-9a87-3d2686e61640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:20.874106 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691cc47-e440-42e4-9a87-3d2686e61640 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:20.874386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691cc47-e440-42e4-9a87-3d2686e61640 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:20.874659 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691cc47-e440-42e4-9a87-3d2686e61640 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:20.887120 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0691cc47-e440-42e4-9a87-3d2686e61640 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:35:20.892149 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-0691cc47-e440-42e4-9a87-3d2686e61640 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:35:20.892149 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:20.892149 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:20.892149 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:20.892149 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:20.892149 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:20.892149 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:20.892149 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:20.892149 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:20.892149 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:20.892149 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:20.892149 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:20.892149 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:35:20.892149 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:35:20.892149 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:35:20.892149 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:35:20.893340 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:35:20.893340 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:35:20.893340 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:35:20.893340 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:35:20.893340 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:20.893340 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:20.893340 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:20.893340 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:20.893340 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:20.893340 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:20.893340 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:20.893340 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:20.893340 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:20.893340 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:20.893340 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:20.893340 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:20.894400 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:20.894400 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:20.894400 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:20.894400 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:20.894400 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:35:20.894400 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:35:20.894400 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:35:20.894400 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:35:20.894400 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:35:20.894400 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:35:20.894400 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:35:20.894400 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:20.897740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-4d3a5a44-ff06-4193-b308-c30effb91714 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 22:35:20.897740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:20.897740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:20.897740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:20.897740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:20.897740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:20.897740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:20.897740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:20.897740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:20.897740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:20.897740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:20.897740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:20.897740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:35:20.897740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:35:20.897740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:35:20.897740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:35:20.898649 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:20.898649 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:20.898649 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:35:20.898649 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:20.898649 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:35:20.898649 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:20.898649 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:20.898649 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:20.898649 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:20.898649 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:20.898649 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:20.898649 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:20.898649 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:20.898649 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:20.898649 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:20.898649 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:20.898649 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:20.898649 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:20.899604 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:20.899604 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:20.899604 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:35:20.899604 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:35:20.899604 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:35:20.899604 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:35:20.899604 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:35:20.899604 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:35:20.899604 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 22:35:20.899604 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:20.900152 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 364/731] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 10 22:35:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:35:20.902002 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667d71e-e11d-4135-b4e4-56aeace95cc7 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:20.902291 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667d71e-e11d-4135-b4e4-56aeace95cc7 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:20.902572 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667d71e-e11d-4135-b4e4-56aeace95cc7 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:20.903603 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 368/732] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:35:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:35:20.909299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 365/733] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:20.910111 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3a5a44-ff06-4193-b308-c30effb91714 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:20.912222 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e667d71e-e11d-4135-b4e4-56aeace95cc7 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:20.964964 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:21.031888 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3a5a44-ff06-4193-b308-c30effb91714 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:21.053660 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e667d71e-e11d-4135-b4e4-56aeace95cc7 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:21.080989 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bbacdb-0695-4c3c-85ce-11c34ab5bb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:21.082565 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bbacdb-0695-4c3c-85ce-11c34ab5bb72 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:21.083153 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bbacdb-0695-4c3c-85ce-11c34ab5bb72 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:21.083628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bbacdb-0695-4c3c-85ce-11c34ab5bb72 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:21.085158 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bbacdb-0695-4c3c-85ce-11c34ab5bb72 None admin] RBAC: Adding query filter params (name=service) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:21.085953 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bbacdb-0695-4c3c-85ce-11c34ab5bb72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:21.087914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bbacdb-0695-4c3c-85ce-11c34ab5bb72 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:21.089732 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ef5bc-937f-4fdf-a1c1-fa66ebe153c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:21.090395 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ef5bc-937f-4fdf-a1c1-fa66ebe153c9 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:21.090674 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ef5bc-937f-4fdf-a1c1-fa66ebe153c9 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:21.090996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ef5bc-937f-4fdf-a1c1-fa66ebe153c9 None admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:21.091718 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3ef5bc-937f-4fdf-a1c1-fa66ebe153c9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:21.092023 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3ef5bc-937f-4fdf-a1c1-fa66ebe153c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:21.093559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3ef5bc-937f-4fdf-a1c1-fa66ebe153c9 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:21.104047 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 369/734] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 10 22:35:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:21.105657 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea76d18-b41d-4c72-b799-40646c92b8b7 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:21.106169 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea76d18-b41d-4c72-b799-40646c92b8b7 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:21.106642 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea76d18-b41d-4c72-b799-40646c92b8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:21.120393 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 366/735] 10.222.0.76 () {56 vars in 1135 bytes} [Wed Aug 10 22:35:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:21.121249 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ef5bc-937f-4fdf-a1c1-fa66ebe153c9 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:21.133030 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-7ea76d18-b41d-4c72-b799-40646c92b8b7 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:21.135380 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-7ea76d18-b41d-4c72-b799-40646c92b8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:21.171650 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:21.237631 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ef5bc-937f-4fdf-a1c1-fa66ebe153c9 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:21.237820 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 370/736] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:21 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:35:21.237820 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea76d18-b41d-4c72-b799-40646c92b8b7 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:21.281918 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e740e0-3618-4143-8d56-6b2ec7e2b4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:21.282849 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e740e0-3618-4143-8d56-6b2ec7e2b4bf None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:21.282966 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e740e0-3618-4143-8d56-6b2ec7e2b4bf None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:21.283351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e740e0-3618-4143-8d56-6b2ec7e2b4bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:21.300996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e740e0-3618-4143-8d56-6b2ec7e2b4bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:21.302382 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:21.304663 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e740e0-3618-4143-8d56-6b2ec7e2b4bf None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:21.329527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-24e740e0-3618-4143-8d56-6b2ec7e2b4bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:21.329527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:21.329527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:21.329527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:21.329527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:21.329527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:21.329527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:21.329527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:21.329527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:21.329527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:21.329527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:21.329527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:21.329527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:21.329527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:21.329527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:21.329527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:21.330995 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:21.330995 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:21.330995 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:21.330995 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:21.330995 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:21.330995 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:21.330995 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:21.330995 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:21.330995 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:21.330995 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:21.330995 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:21.330995 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:21.330995 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:21.330995 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:21.330995 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:21.330995 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:21.330995 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:21.332261 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:21.332261 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:21.332261 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:21.333686 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 367/737] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:21.334625 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e740e0-3618-4143-8d56-6b2ec7e2b4bf None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:21.380310 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea76d18-b41d-4c72-b799-40646c92b8b7 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:21.414454 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc98a7-fa7f-42c2-be33-e8143f20bc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:21.415788 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc98a7-fa7f-42c2-be33-e8143f20bc51 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:21.416261 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc98a7-fa7f-42c2-be33-e8143f20bc51 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:21.416537 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc98a7-fa7f-42c2-be33-e8143f20bc51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:21.447178 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:21.451331 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e740e0-3618-4143-8d56-6b2ec7e2b4bf None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:21.487882 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955033b1-59cb-4f4a-b684-98a200aee3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:21.488509 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955033b1-59cb-4f4a-b684-98a200aee3f1 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:21.488803 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955033b1-59cb-4f4a-b684-98a200aee3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:21.489103 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955033b1-59cb-4f4a-b684-98a200aee3f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:21.492561 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edc98a7-fa7f-42c2-be33-e8143f20bc51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:21.494060 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edc98a7-fa7f-42c2-be33-e8143f20bc51 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:21.500137 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955033b1-59cb-4f4a-b684-98a200aee3f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:35:21.506298 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-955033b1-59cb-4f4a-b684-98a200aee3f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:35:21.506298 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:21.506298 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:21.506298 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:21.506298 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:21.506298 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:21.506298 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:21.506298 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:21.506298 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:21.506298 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:21.506298 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:21.506298 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:21.506298 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:35:21.506298 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:35:21.506298 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:35:21.506298 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:35:21.507265 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:35:21.507265 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:35:21.507265 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:35:21.507265 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:35:21.507265 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:21.507265 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:21.507265 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:21.507265 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:21.507265 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:21.507265 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:21.507265 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:21.507265 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:21.507265 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:21.507265 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:21.507265 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:21.507265 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:21.508127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:21.508127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:21.508127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:21.508127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:21.508127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:35:21.508127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:35:21.508127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:35:21.508127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:35:21.508127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:35:21.508127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:35:21.508127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:35:21.508127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:21.508127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 368/738] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:35:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:21.509208 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedaa672-51a6-43c1-9646-6c746c2109ab None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:21.509208 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedaa672-51a6-43c1-9646-6c746c2109ab None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:21.509208 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedaa672-51a6-43c1-9646-6c746c2109ab None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:21.513308 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 369/739] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:21.513308 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedaa672-51a6-43c1-9646-6c746c2109ab None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:21.545076 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:21.620090 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 371/740] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:35:21 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 22:35:21.620090 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc98a7-fa7f-42c2-be33-e8143f20bc51 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:21.626932 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedaa672-51a6-43c1-9646-6c746c2109ab None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:21.670183 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c6f581-b7e6-4ab1-8e15-760c6f01ea8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:21.670799 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6f581-b7e6-4ab1-8e15-760c6f01ea8b None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:21.671084 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6f581-b7e6-4ab1-8e15-760c6f01ea8b None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:21.671360 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6f581-b7e6-4ab1-8e15-760c6f01ea8b None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:21.672003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c6f581-b7e6-4ab1-8e15-760c6f01ea8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:21.672315 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c6f581-b7e6-4ab1-8e15-760c6f01ea8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:21.674771 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c6f581-b7e6-4ab1-8e15-760c6f01ea8b None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:21.686378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 370/741] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:21.687519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6246af-278d-4cc1-aaad-3b03a65dcd76 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:21.687814 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6246af-278d-4cc1-aaad-3b03a65dcd76 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:21.688088 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6246af-278d-4cc1-aaad-3b03a65dcd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:21.709026 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-5c6246af-278d-4cc1-aaad-3b03a65dcd76 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:21.710268 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-5c6246af-278d-4cc1-aaad-3b03a65dcd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:21.725960 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:21.733045 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc98a7-fa7f-42c2-be33-e8143f20bc51 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:21.771052 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187303c1-0f01-44b3-9986-c30b47ec1376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:21.771665 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187303c1-0f01-44b3-9986-c30b47ec1376 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:21.772065 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187303c1-0f01-44b3-9986-c30b47ec1376 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:21.772337 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187303c1-0f01-44b3-9986-c30b47ec1376 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:21.772994 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187303c1-0f01-44b3-9986-c30b47ec1376 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:21.773308 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187303c1-0f01-44b3-9986-c30b47ec1376 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:21.774684 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187303c1-0f01-44b3-9986-c30b47ec1376 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:21.784417 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 372/742] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:35:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:21.784900 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187303c1-0f01-44b3-9986-c30b47ec1376 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:21.787546 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:21.792244 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 371/743] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:21 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 22:35:21.793313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6246af-278d-4cc1-aaad-3b03a65dcd76 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:21.899432 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187303c1-0f01-44b3-9986-c30b47ec1376 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:21.906899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6246af-278d-4cc1-aaad-3b03a65dcd76 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:21.934879 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0442d8-4fb9-430e-a636-5f2dffa67536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:21.935505 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0442d8-4fb9-430e-a636-5f2dffa67536 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:21.935779 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0442d8-4fb9-430e-a636-5f2dffa67536 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:21.936050 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0442d8-4fb9-430e-a636-5f2dffa67536 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:21.943664 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8edb75-86fc-4496-82d5-70a5d1b59497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:21.944489 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8edb75-86fc-4496-82d5-70a5d1b59497 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:21.944997 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8edb75-86fc-4496-82d5-70a5d1b59497 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:21.945453 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8edb75-86fc-4496-82d5-70a5d1b59497 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:21.946660 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0442d8-4fb9-430e-a636-5f2dffa67536 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:21.948368 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0442d8-4fb9-430e-a636-5f2dffa67536 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:21.953699 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8edb75-86fc-4496-82d5-70a5d1b59497 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:21.956116 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8edb75-86fc-4496-82d5-70a5d1b59497 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:21.963039 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-bc0442d8-4fb9-430e-a636-5f2dffa67536 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:21.963039 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:21.963039 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:21.963039 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:21.963039 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:21.963039 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:21.963039 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:21.963039 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:21.963039 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:21.963039 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:21.963039 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:21.963039 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:21.963039 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:21.963039 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:21.963039 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:21.963039 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:21.964160 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:21.964160 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:21.964160 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:21.964160 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:21.964160 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:21.964160 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:21.964160 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:21.964160 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:21.964160 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:21.964160 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:21.964160 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:21.964160 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:21.964160 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:21.964160 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:21.964160 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:21.964160 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:21.964160 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:21.965067 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:21.965067 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:21.965067 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:21.965067 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 373/744] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:21.965455 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0442d8-4fb9-430e-a636-5f2dffa67536 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:21.974859 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-9f8edb75-86fc-4496-82d5-70a5d1b59497 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 22:35:21.974859 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:21.974859 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:21.974859 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:21.974859 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:21.974859 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:21.974859 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:21.974859 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:21.974859 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:21.974859 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:21.974859 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:21.974859 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:21.974859 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:35:21.974859 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:35:21.974859 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:35:21.974859 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:35:21.976041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:21.976041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:21.976041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:35:21.976041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:21.976041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:35:21.976041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:21.976041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:21.976041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:21.976041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:21.976041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:21.976041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:21.976041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:21.976041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:21.976041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:21.976041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:21.976041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:21.976041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:21.976041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:21.977228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:21.977228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:21.977228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:35:21.977228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:35:21.977228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:35:21.977228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:35:21.977228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:35:21.977228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:35:21.977228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 22:35:21.977228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:21.979909 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 372/745] 10.222.0.76 () {56 vars in 1117 bytes} [Wed Aug 10 22:35:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:21.981268 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8edb75-86fc-4496-82d5-70a5d1b59497 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:21.993500 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:22.034995 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:22.081595 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0442d8-4fb9-430e-a636-5f2dffa67536 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:22.093916 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8edb75-86fc-4496-82d5-70a5d1b59497 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:22.116808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97944cb-1b2e-493f-b85e-0fa4e4637c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:22.117481 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97944cb-1b2e-493f-b85e-0fa4e4637c71 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:22.117764 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97944cb-1b2e-493f-b85e-0fa4e4637c71 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:22.118038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97944cb-1b2e-493f-b85e-0fa4e4637c71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:22.129450 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97944cb-1b2e-493f-b85e-0fa4e4637c71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:22.130903 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97944cb-1b2e-493f-b85e-0fa4e4637c71 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:22.131181 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f574d3a4-f70c-4f0f-8ba1-a54ef0d0335d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:22.131789 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574d3a4-f70c-4f0f-8ba1-a54ef0d0335d None admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:22.132068 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574d3a4-f70c-4f0f-8ba1-a54ef0d0335d None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:22.132338 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574d3a4-f70c-4f0f-8ba1-a54ef0d0335d None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:22.133089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f574d3a4-f70c-4f0f-8ba1-a54ef0d0335d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:22.134588 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f574d3a4-f70c-4f0f-8ba1-a54ef0d0335d None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:22.145834 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-d97944cb-1b2e-493f-b85e-0fa4e4637c71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:22.145834 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:22.145834 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:22.145834 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:22.145834 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:22.145834 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:22.145834 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:22.145834 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:22.145834 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:22.145834 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:22.145834 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:22.145834 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:22.145834 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:22.145834 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:22.145834 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:22.145834 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:22.147080 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:22.147080 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:22.147080 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:22.147080 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:22.147080 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:22.147080 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:22.147080 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:22.147080 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:22.147080 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:22.147080 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:22.147080 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:22.147080 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:22.147080 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:22.147080 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:22.147080 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:22.147080 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:22.147080 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:22.148576 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:22.148576 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:22.148576 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:22.148576 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 374/746] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:22.148576 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97944cb-1b2e-493f-b85e-0fa4e4637c71 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:22.165353 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 373/747] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 10 22:35:21 2022] POST /identity/v3/services => generated 240 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:35:22.165353 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f574d3a4-f70c-4f0f-8ba1-a54ef0d0335d None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:22.237387 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:22.277616 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97944cb-1b2e-493f-b85e-0fa4e4637c71 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:22.285026 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f574d3a4-f70c-4f0f-8ba1-a54ef0d0335d None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:22.286954 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:22.319233 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36a694b-bb6a-4ebd-a4c1-036bb8c2d548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:22.320261 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a694b-bb6a-4ebd-a4c1-036bb8c2d548 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:22.320542 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a694b-bb6a-4ebd-a4c1-036bb8c2d548 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:22.320813 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a694b-bb6a-4ebd-a4c1-036bb8c2d548 None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:22.321514 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36a694b-bb6a-4ebd-a4c1-036bb8c2d548 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:22.321807 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36a694b-bb6a-4ebd-a4c1-036bb8c2d548 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:22.323468 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36a694b-bb6a-4ebd-a4c1-036bb8c2d548 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:22.327617 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d1e034-d6c4-4b1e-b908-97fd6b290255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:22.328602 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d1e034-d6c4-4b1e-b908-97fd6b290255 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:22.329091 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d1e034-d6c4-4b1e-b908-97fd6b290255 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:22.329643 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d1e034-d6c4-4b1e-b908-97fd6b290255 None admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:22.331039 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d1e034-d6c4-4b1e-b908-97fd6b290255 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:22.332260 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d1e034-d6c4-4b1e-b908-97fd6b290255 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:22.334613 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d1e034-d6c4-4b1e-b908-97fd6b290255 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:22.337009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 375/748] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:22.337816 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36a694b-bb6a-4ebd-a4c1-036bb8c2d548 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:22.359685 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 374/749] 10.222.0.76 () {56 vars in 1149 bytes} [Wed Aug 10 22:35:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:22.361840 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3607c89-23c6-4c89-9f81-73770a7b8b44 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:22.362345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3607c89-23c6-4c89-9f81-73770a7b8b44 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:22.362681 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3607c89-23c6-4c89-9f81-73770a7b8b44 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:22.363917 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 375/750] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:22.364651 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3607c89-23c6-4c89-9f81-73770a7b8b44 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:22.452400 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36a694b-bb6a-4ebd-a4c1-036bb8c2d548 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:22.476418 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3607c89-23c6-4c89-9f81-73770a7b8b44 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:22.490021 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d60257-5b87-41de-880b-eca70fecfd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:22.490839 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d60257-5b87-41de-880b-eca70fecfd68 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:22.491267 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d60257-5b87-41de-880b-eca70fecfd68 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:22.491682 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d60257-5b87-41de-880b-eca70fecfd68 None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:22.492490 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d60257-5b87-41de-880b-eca70fecfd68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:22.492938 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d60257-5b87-41de-880b-eca70fecfd68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:22.494490 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d60257-5b87-41de-880b-eca70fecfd68 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:22.502158 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:22.511243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 376/751] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:22.511243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d60257-5b87-41de-880b-eca70fecfd68 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:22.520354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dcaf0f-0d79-459f-9f4f-1af0ca42d38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:22.521021 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dcaf0f-0d79-459f-9f4f-1af0ca42d38c None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:22.521317 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dcaf0f-0d79-459f-9f4f-1af0ca42d38c None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:22.521597 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dcaf0f-0d79-459f-9f4f-1af0ca42d38c None admin] PATH_INFO: `/v3/projects/service` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:22.528805 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dcaf0f-0d79-459f-9f4f-1af0ca42d38c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:22.530521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dcaf0f-0d79-459f-9f4f-1af0ca42d38c None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:22.539578 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:22.543516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-e0dcaf0f-0d79-459f-9f4f-1af0ca42d38c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:22.543516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:22.543516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:22.543516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:22.543516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:22.543516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:22.543516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:22.543516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:22.543516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:22.543516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:22.543516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:22.543516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:22.543516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:35:22.543516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:35:22.543516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:35:22.544408 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:35:22.544408 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:22.544408 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:22.544408 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:22.544408 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:22.544408 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:22.544408 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:22.544408 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:22.544408 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:22.544408 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:22.544408 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:22.544408 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:22.544408 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:22.544408 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:22.544408 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:22.544408 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:22.544408 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:22.545307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:35:22.545307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:35:22.545307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:35:22.545307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:35:22.545307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:35:22.545307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:35:22.545307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:22.545307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:22.547303 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 376/752] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:35:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:22.548898 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ee893-a383-4597-b927-4970e1603533 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:22.549355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ee893-a383-4597-b927-4970e1603533 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:22.549792 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ee893-a383-4597-b927-4970e1603533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:22.568185 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-945ee893-a383-4597-b927-4970e1603533 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:22.569425 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-945ee893-a383-4597-b927-4970e1603533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:22.634271 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d60257-5b87-41de-880b-eca70fecfd68 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:22.653690 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 377/753] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:35:22.653690 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ee893-a383-4597-b927-4970e1603533 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:22.688482 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab97244-7574-4004-8b7c-ef4b0ada96b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:22.689294 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab97244-7574-4004-8b7c-ef4b0ada96b1 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:22.689582 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab97244-7574-4004-8b7c-ef4b0ada96b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:22.689969 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab97244-7574-4004-8b7c-ef4b0ada96b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:22.761688 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ee893-a383-4597-b927-4970e1603533 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:22.766661 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab97244-7574-4004-8b7c-ef4b0ada96b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:22.768920 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab97244-7574-4004-8b7c-ef4b0ada96b1 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:22.776388 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:22.798319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:22.801441 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6301fb0-4911-46c8-94d2-9404f9d0cc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:22.802088 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6301fb0-4911-46c8-94d2-9404f9d0cc0e None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:22.802372 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6301fb0-4911-46c8-94d2-9404f9d0cc0e None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:22.802645 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6301fb0-4911-46c8-94d2-9404f9d0cc0e None admin] PATH_INFO: `/v3/users/nova` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:22.810053 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6301fb0-4911-46c8-94d2-9404f9d0cc0e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:22.811508 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6301fb0-4911-46c8-94d2-9404f9d0cc0e None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:22.823435 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-c6301fb0-4911-46c8-94d2-9404f9d0cc0e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:35:22.823435 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:22.823435 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:22.823435 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:22.823435 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:22.823435 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:22.823435 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:22.823435 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:22.823435 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:22.823435 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:22.823435 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:22.823435 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:22.823435 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:35:22.823435 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:35:22.823435 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:35:22.823435 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:35:22.824362 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:22.824362 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:22.824362 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:35:22.824362 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:22.824362 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:35:22.824362 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:22.824362 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:22.824362 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:22.824362 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:22.824362 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:22.824362 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:22.824362 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:22.824362 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:22.824362 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:22.824362 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:22.824362 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:22.824362 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:22.824362 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:22.825975 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:22.825975 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:22.825975 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:35:22.825975 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:35:22.825975 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:35:22.825975 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:35:22.825975 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:35:22.825975 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:35:22.825975 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:35:22.825975 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:22.825975 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 378/754] 10.222.0.76 () {56 vars in 1093 bytes} [Wed Aug 10 22:35:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:22.825975 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6301fb0-4911-46c8-94d2-9404f9d0cc0e None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:22.883100 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 377/755] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:35:22 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 22:35:22.883961 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab97244-7574-4004-8b7c-ef4b0ada96b1 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:22.938341 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6301fb0-4911-46c8-94d2-9404f9d0cc0e None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:22.973863 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eae6b4-1110-422e-9467-6f933721951c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:22.974474 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eae6b4-1110-422e-9467-6f933721951c None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:22.974754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eae6b4-1110-422e-9467-6f933721951c None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:22.975033 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eae6b4-1110-422e-9467-6f933721951c None admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:22.975727 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eae6b4-1110-422e-9467-6f933721951c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:22.976030 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eae6b4-1110-422e-9467-6f933721951c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:22.977492 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eae6b4-1110-422e-9467-6f933721951c None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:22.992418 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 379/756] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:35:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:22.993273 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eae6b4-1110-422e-9467-6f933721951c None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:23.002999 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab97244-7574-4004-8b7c-ef4b0ada96b1 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:23.035375 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:23.035600 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:23.049115 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e800775-2f5c-41be-916e-e58df23eb7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:23.049818 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e800775-2f5c-41be-916e-e58df23eb7c2 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:23.050124 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e800775-2f5c-41be-916e-e58df23eb7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:23.050372 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e800775-2f5c-41be-916e-e58df23eb7c2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:23.050957 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e800775-2f5c-41be-916e-e58df23eb7c2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:23.052323 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e800775-2f5c-41be-916e-e58df23eb7c2 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:23.063648 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-3e800775-2f5c-41be-916e-e58df23eb7c2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 22:35:23.063648 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:23.063648 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:35:23.063648 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:35:23.063648 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:35:23.063648 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:35:23.063648 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:35:23.063648 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:23.063648 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 22:35:23.063648 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:23.063648 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:35:23.063648 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:23.063648 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:23.063648 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:23.063648 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:23.063648 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:23.063648 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:23.065184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:23.065184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:23.065184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:23.065184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:23.065184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:23.065184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:23.065184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:35:23.065184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:35:23.065184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:35:23.065184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:35:23.065184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:23.065184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:23.065184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:23.065184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:23.065184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:23.065184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:23.066387 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:23.066387 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:23.066387 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:23.066387 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:23.066387 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:23.066387 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:23.066387 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:23.066387 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:23.066387 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:23.066387 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:23.066387 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:35:23.066387 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:23.066387 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 22:35:23.066387 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:23.066387 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 378/757] 10.222.0.76 () {56 vars in 1129 bytes} [Wed Aug 10 22:35:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:23.067549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e800775-2f5c-41be-916e-e58df23eb7c2 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:23.115548 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eae6b4-1110-422e-9467-6f933721951c None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:23.152520 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5623b-90bc-4ba3-82a2-70d72243bdbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:23.153134 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5623b-90bc-4ba3-82a2-70d72243bdbb None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:23.153441 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5623b-90bc-4ba3-82a2-70d72243bdbb None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:23.153716 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5623b-90bc-4ba3-82a2-70d72243bdbb None admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:23.154437 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5623b-90bc-4ba3-82a2-70d72243bdbb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:23.154741 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5623b-90bc-4ba3-82a2-70d72243bdbb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:23.156173 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5623b-90bc-4ba3-82a2-70d72243bdbb None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:23.177293 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e800775-2f5c-41be-916e-e58df23eb7c2 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:23.183656 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 380/758] 10.222.0.76 () {56 vars in 1133 bytes} [Wed Aug 10 22:35:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:23.183656 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5623b-90bc-4ba3-82a2-70d72243bdbb None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:23.214486 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439882f1-4020-46fc-9458-ff69694ad936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:23.215125 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439882f1-4020-46fc-9458-ff69694ad936 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:23.215405 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439882f1-4020-46fc-9458-ff69694ad936 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:23.215750 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439882f1-4020-46fc-9458-ff69694ad936 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:23.226954 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439882f1-4020-46fc-9458-ff69694ad936 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:23.228399 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439882f1-4020-46fc-9458-ff69694ad936 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:23.243571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-439882f1-4020-46fc-9458-ff69694ad936 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:23.243571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:23.243571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:23.243571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:23.243571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:23.243571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:23.243571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:23.243571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:23.243571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:23.243571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:23.243571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:23.243571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:23.243571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:23.243571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:23.243571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:23.243571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:23.244544 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:23.244544 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:23.244544 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:23.244544 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:23.244544 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:23.244544 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:23.244544 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:23.244544 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:23.244544 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:23.244544 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:23.244544 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:23.244544 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:23.244544 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:23.244544 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:23.244544 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:23.244544 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:23.244544 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:23.245755 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:23.245755 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:23.245755 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:23.245755 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 379/759] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:23.245755 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439882f1-4020-46fc-9458-ff69694ad936 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:23.292977 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:23.295710 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:23.306577 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5623b-90bc-4ba3-82a2-70d72243bdbb None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:23.345650 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebad8ce-b658-422c-a271-ef7c8238b51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:23.346690 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebad8ce-b658-422c-a271-ef7c8238b51c None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:23.347177 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebad8ce-b658-422c-a271-ef7c8238b51c None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:23.347650 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebad8ce-b658-422c-a271-ef7c8238b51c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:23.348996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebad8ce-b658-422c-a271-ef7c8238b51c None admin] RBAC: Adding query filter params (role.id=6641e95fdc1f4c098ebc372bb692d227, scope.project.id=3caa2ea846f7449b8c483788fc5846c8, user.id=1e3a76df95a34b9fb9816f22b85d1071) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:23.349334 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebad8ce-b658-422c-a271-ef7c8238b51c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:23.350980 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebad8ce-b658-422c-a271-ef7c8238b51c None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:23.362302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 381/760] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:35:23 2022] GET /identity/v3/role_assignments?user.id=1e3a76df95a34b9fb9816f22b85d1071&scope.project.id=3caa2ea846f7449b8c483788fc5846c8&role.id=6641e95fdc1f4c098ebc372bb692d227 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:23.363929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebad8ce-b658-422c-a271-ef7c8238b51c None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:23.369208 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439882f1-4020-46fc-9458-ff69694ad936 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:23.412396 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e4da9-ed73-4760-9b87-43df2e9ad0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:23.413029 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e4da9-ed73-4760-9b87-43df2e9ad0ca None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:23.413356 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e4da9-ed73-4760-9b87-43df2e9ad0ca None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:23.413638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e4da9-ed73-4760-9b87-43df2e9ad0ca None admin] PATH_INFO: `/v3/services` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:23.414401 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1e4da9-ed73-4760-9b87-43df2e9ad0ca None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:23.414711 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1e4da9-ed73-4760-9b87-43df2e9ad0ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:23.416331 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1e4da9-ed73-4760-9b87-43df2e9ad0ca None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:23.425710 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 380/761] 10.222.0.76 () {56 vars in 1124 bytes} [Wed Aug 10 22:35:23 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:23.426664 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e4da9-ed73-4760-9b87-43df2e9ad0ca None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:23.521475 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebad8ce-b658-422c-a271-ef7c8238b51c None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:23.551239 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e4da9-ed73-4760-9b87-43df2e9ad0ca None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:23.558310 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:23.568813 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b6ef05-5757-4bc0-bf74-99e41476a210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:23.570011 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b6ef05-5757-4bc0-bf74-99e41476a210 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:23.570490 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b6ef05-5757-4bc0-bf74-99e41476a210 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:23.570965 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b6ef05-5757-4bc0-bf74-99e41476a210 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:23.585382 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b6ef05-5757-4bc0-bf74-99e41476a210 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:23.587276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b6ef05-5757-4bc0-bf74-99e41476a210 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:23.587386 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e04f84-5f50-4c63-9918-fe79f792541a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:23.588018 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e04f84-5f50-4c63-9918-fe79f792541a None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:23.588355 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e04f84-5f50-4c63-9918-fe79f792541a None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:23.588686 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e04f84-5f50-4c63-9918-fe79f792541a None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:23.589439 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e04f84-5f50-4c63-9918-fe79f792541a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:23.589805 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e04f84-5f50-4c63-9918-fe79f792541a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:23.591227 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e04f84-5f50-4c63-9918-fe79f792541a None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:23.607662 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 381/762] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:23.607662 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e04f84-5f50-4c63-9918-fe79f792541a None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:23.607909 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-82b6ef05-5757-4bc0-bf74-99e41476a210 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:23.607909 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:23.607909 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:23.607909 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:23.607909 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:23.607909 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:23.607909 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:23.607909 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:23.607909 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:23.607909 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:23.607909 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:23.607909 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:23.607909 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:23.607909 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:23.607909 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:23.607909 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:23.608830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:23.608830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:23.608830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:23.608830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:23.608830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:23.608830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:23.608830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:23.608830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:23.608830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:23.608830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:23.608830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:23.608830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:23.608830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:23.608830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:23.608830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:23.608830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:23.608830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:23.609754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:23.609754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:23.609754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:23.609754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 382/763] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:23.614092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b6ef05-5757-4bc0-bf74-99e41476a210 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:23.627098 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:23.731440 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e04f84-5f50-4c63-9918-fe79f792541a None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:23.739431 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b6ef05-5757-4bc0-bf74-99e41476a210 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:23.770960 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff18f0-db2e-4e6c-b760-24e881dabe9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:23.771931 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff18f0-db2e-4e6c-b760-24e881dabe9e None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:23.772387 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff18f0-db2e-4e6c-b760-24e881dabe9e None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:23.772872 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff18f0-db2e-4e6c-b760-24e881dabe9e None admin] PATH_INFO: `/v3/services` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:23.773726 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff18f0-db2e-4e6c-b760-24e881dabe9e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:23.774217 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff18f0-db2e-4e6c-b760-24e881dabe9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:23.775743 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803c68d8-ac35-40be-a20e-94b32d81dc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:23.775932 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff18f0-db2e-4e6c-b760-24e881dabe9e None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:23.776490 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803c68d8-ac35-40be-a20e-94b32d81dc55 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:23.776837 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803c68d8-ac35-40be-a20e-94b32d81dc55 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:23.777176 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803c68d8-ac35-40be-a20e-94b32d81dc55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:23.782206 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 382/764] 10.222.0.76 () {56 vars in 1124 bytes} [Wed Aug 10 22:35:23 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:23.784039 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff18f0-db2e-4e6c-b760-24e881dabe9e None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:23.785295 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803c68d8-ac35-40be-a20e-94b32d81dc55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:23.786828 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803c68d8-ac35-40be-a20e-94b32d81dc55 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:23.799004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-803c68d8-ac35-40be-a20e-94b32d81dc55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:23.799004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:23.799004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:23.799004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:23.799004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:23.799004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:23.799004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:23.799004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:23.799004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:23.799004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:23.799004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:23.799004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:23.799004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:35:23.799004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:35:23.799004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:35:23.799958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:35:23.799958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:23.799958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:23.799958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:23.799958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:23.799958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:23.799958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:23.799958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:23.799958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:23.799958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:23.799958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:23.799958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:23.799958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:23.799958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:23.799958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:23.799958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:23.799958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:23.801176 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:35:23.801176 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:35:23.801176 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:35:23.801176 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:35:23.801176 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:35:23.801176 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:35:23.801176 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:23.801176 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:23.805112 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 383/765] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:35:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:23.806029 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803c68d8-ac35-40be-a20e-94b32d81dc55 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:23.821704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:23.881896 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:23.902116 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff18f0-db2e-4e6c-b760-24e881dabe9e None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:23.920717 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803c68d8-ac35-40be-a20e-94b32d81dc55 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:23.940501 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d774f-ce65-4323-9fe0-805d5b6ee9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:23.941560 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d774f-ce65-4323-9fe0-805d5b6ee9b6 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:23.942018 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d774f-ce65-4323-9fe0-805d5b6ee9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:23.942486 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d774f-ce65-4323-9fe0-805d5b6ee9b6 None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:23.943299 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d774f-ce65-4323-9fe0-805d5b6ee9b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:23.943862 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d774f-ce65-4323-9fe0-805d5b6ee9b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:23.945590 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d774f-ce65-4323-9fe0-805d5b6ee9b6 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:23.959545 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 383/766] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:23.960791 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d774f-ce65-4323-9fe0-805d5b6ee9b6 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:23.965365 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f13d2-82ca-4d04-9b58-77eeedd479c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:23.966067 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f13d2-82ca-4d04-9b58-77eeedd479c2 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:23.966415 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f13d2-82ca-4d04-9b58-77eeedd479c2 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:23.966776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f13d2-82ca-4d04-9b58-77eeedd479c2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:23.967679 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50f13d2-82ca-4d04-9b58-77eeedd479c2 None admin] RBAC: Adding query filter params (interface=public, service_id=d5424fc3303748b5892731dbe4a14d15, region_id=RegionOne) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:23.968048 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50f13d2-82ca-4d04-9b58-77eeedd479c2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:23.969626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50f13d2-82ca-4d04-9b58-77eeedd479c2 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:23.979233 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 384/767] 10.222.0.76 () {56 vars in 1251 bytes} [Wed Aug 10 22:35:23 2022] GET /identity/v3/endpoints?service_id=d5424fc3303748b5892731dbe4a14d15&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:23.982300 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f13d2-82ca-4d04-9b58-77eeedd479c2 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:24.071644 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:24.083125 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d774f-ce65-4323-9fe0-805d5b6ee9b6 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:24.102252 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f13d2-82ca-4d04-9b58-77eeedd479c2 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:24.122377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51991309-dc10-462b-8d8c-4d071e5c696a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:24.123273 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51991309-dc10-462b-8d8c-4d071e5c696a None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:24.123728 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51991309-dc10-462b-8d8c-4d071e5c696a None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:24.124168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51991309-dc10-462b-8d8c-4d071e5c696a None admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:24.125155 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51991309-dc10-462b-8d8c-4d071e5c696a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:24.125659 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51991309-dc10-462b-8d8c-4d071e5c696a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:24.128193 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51991309-dc10-462b-8d8c-4d071e5c696a None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:24.134447 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:24.140495 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac6f8f-6e06-4f88-9423-64e483c75a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:24.141247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac6f8f-6e06-4f88-9423-64e483c75a04 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:24.142453 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac6f8f-6e06-4f88-9423-64e483c75a04 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:24.142767 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac6f8f-6e06-4f88-9423-64e483c75a04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:24.145981 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 384/768] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:35:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:24.147179 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51991309-dc10-462b-8d8c-4d071e5c696a None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:24.155754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac6f8f-6e06-4f88-9423-64e483c75a04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:24.158364 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac6f8f-6e06-4f88-9423-64e483c75a04 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:24.175521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-53ac6f8f-6e06-4f88-9423-64e483c75a04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:24.175521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:24.175521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:24.175521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:24.175521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:24.175521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:24.175521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:24.175521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:24.175521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:24.175521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:24.175521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:24.175521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:24.175521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:35:24.175521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:35:24.175521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:35:24.176440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:35:24.176440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:24.176440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:24.176440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:24.176440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:24.176440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:24.176440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:24.176440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:24.176440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:24.176440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:24.176440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:24.176440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:24.176440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:24.176440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:24.176440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:24.176440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:24.176440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:24.177345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:35:24.177345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:35:24.177345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:35:24.177345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:35:24.177345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:35:24.177345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:35:24.177345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:24.177345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:24.181342 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 385/769] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:35:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:24.182380 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac6f8f-6e06-4f88-9423-64e483c75a04 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:24.281243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51991309-dc10-462b-8d8c-4d071e5c696a None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:24.308692 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac6f8f-6e06-4f88-9423-64e483c75a04 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:24.328172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f2d9f6-a8be-4cd1-8a9f-e6ab9140bef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:24.329435 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2d9f6-a8be-4cd1-8a9f-e6ab9140bef7 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:24.329716 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2d9f6-a8be-4cd1-8a9f-e6ab9140bef7 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:24.331535 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2d9f6-a8be-4cd1-8a9f-e6ab9140bef7 None admin] PATH_INFO: `/v3/services` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:24.333361 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f2d9f6-a8be-4cd1-8a9f-e6ab9140bef7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:24.334161 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f2d9f6-a8be-4cd1-8a9f-e6ab9140bef7 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:24.346033 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 385/770] 10.222.0.76 () {56 vars in 1087 bytes} [Wed Aug 10 22:35:24 2022] GET /identity/v3/services => generated 772 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:24.346033 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f2d9f6-a8be-4cd1-8a9f-e6ab9140bef7 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:24.351585 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f45b8-d211-4a44-92d2-d9efd914e5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:24.352207 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f45b8-d211-4a44-92d2-d9efd914e5c8 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:24.352485 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f45b8-d211-4a44-92d2-d9efd914e5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:24.352753 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f45b8-d211-4a44-92d2-d9efd914e5c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:24.353935 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f45b8-d211-4a44-92d2-d9efd914e5c8 None admin] RBAC: Adding query filter params (role.id=5dfd0f952cd641bbb2257860c4c892ac, scope.project.id=3caa2ea846f7449b8c483788fc5846c8, user.id=aff5e804c64345b298c745287a1d0a32) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:24.354387 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f45b8-d211-4a44-92d2-d9efd914e5c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:24.355864 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f45b8-d211-4a44-92d2-d9efd914e5c8 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:24.364063 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 386/771] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:35:24 2022] GET /identity/v3/role_assignments?user.id=aff5e804c64345b298c745287a1d0a32&scope.project.id=3caa2ea846f7449b8c483788fc5846c8&role.id=5dfd0f952cd641bbb2257860c4c892ac => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:24.365570 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b922f91-d97c-4408-a961-a3bc9576e35e None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:24.365901 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b922f91-d97c-4408-a961-a3bc9576e35e None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:24.366180 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b922f91-d97c-4408-a961-a3bc9576e35e None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:24.367900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 387/772] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:24.377316 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:24.377545 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f27c0-1b0b-4af1-aba5-345e38bebe9b None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:24.377545 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f27c0-1b0b-4af1-aba5-345e38bebe9b None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:24.377545 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f27c0-1b0b-4af1-aba5-345e38bebe9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:24.410449 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-308f27c0-1b0b-4af1-aba5-345e38bebe9b None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:24.411715 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-308f27c0-1b0b-4af1-aba5-345e38bebe9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:24.498279 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:24.523559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 388/773] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:35:24.553239 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f27c0-1b0b-4af1-aba5-345e38bebe9b None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:24.586491 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f2d9f6-a8be-4cd1-8a9f-e6ab9140bef7 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:24.625710 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53f0136-ecf6-4c36-90d1-9c3750c2f1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:24.626405 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f0136-ecf6-4c36-90d1-9c3750c2f1b7 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:24.626682 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f0136-ecf6-4c36-90d1-9c3750c2f1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:24.627041 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f0136-ecf6-4c36-90d1-9c3750c2f1b7 None admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:24.627739 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53f0136-ecf6-4c36-90d1-9c3750c2f1b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:24.628149 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53f0136-ecf6-4c36-90d1-9c3750c2f1b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:24.629699 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53f0136-ecf6-4c36-90d1-9c3750c2f1b7 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:24.643416 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 386/774] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:35:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:24.649234 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53f0136-ecf6-4c36-90d1-9c3750c2f1b7 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:24.683317 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f27c0-1b0b-4af1-aba5-345e38bebe9b None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:24.740784 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e4c961-571f-4f42-8d4f-59f3c0ca72df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:24.742878 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4c961-571f-4f42-8d4f-59f3c0ca72df None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:24.743224 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4c961-571f-4f42-8d4f-59f3c0ca72df None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:24.743502 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4c961-571f-4f42-8d4f-59f3c0ca72df None admin] PATH_INFO: `/v3/services/network` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:24.745027 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e4c961-571f-4f42-8d4f-59f3c0ca72df None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:24.745225 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:24.746655 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e4c961-571f-4f42-8d4f-59f3c0ca72df None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:24.766089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-e2e4c961-571f-4f42-8d4f-59f3c0ca72df None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 22:35:24.766089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:24.766089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:35:24.766089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:35:24.766089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:35:24.766089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:35:24.766089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:35:24.766089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:24.766089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 22:35:24.766089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:24.766089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:35:24.766089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:24.766089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:24.766089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:24.766089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:24.766089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:24.766089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:24.767849 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:24.767849 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:24.767849 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:24.767849 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:24.767849 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:24.767849 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:24.767849 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:35:24.767849 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:35:24.767849 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:35:24.767849 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:35:24.767849 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:24.767849 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:24.767849 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:24.767849 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:24.767849 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:24.767849 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:24.768663 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:24.768663 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:24.768663 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:24.768663 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:24.768663 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:24.768663 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:24.768663 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:24.768663 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:24.768663 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:24.768663 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:24.768663 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:35:24.768663 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:24.768663 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 22:35:24.768663 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:24.769428 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 389/775] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:35:24 2022] GET /identity/v3/services/network => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:35:24.777452 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e4c961-571f-4f42-8d4f-59f3c0ca72df None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:24.798950 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53f0136-ecf6-4c36-90d1-9c3750c2f1b7 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:24.862834 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:24.870517 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ce7ed-0b80-4ab6-9667-2a68d8444705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:24.871193 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ce7ed-0b80-4ab6-9667-2a68d8444705 None admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:24.871512 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ce7ed-0b80-4ab6-9667-2a68d8444705 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:24.871817 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ce7ed-0b80-4ab6-9667-2a68d8444705 None admin] PATH_INFO: `/v3/projects/3caa2ea846f7449b8c483788fc5846c8/users/b22838a8ad3949e59cdbaaa1801cd07f/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:24.873911 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ce7ed-0b80-4ab6-9667-2a68d8444705 None admin] RBAC: Authorizing `identity:create_grant(project_id=3caa2ea846f7449b8c483788fc5846c8, user_id=b22838a8ad3949e59cdbaaa1801cd07f, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:24.875343 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ce7ed-0b80-4ab6-9667-2a68d8444705 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:24.902447 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 387/776] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 10 22:35:24 2022] PUT /identity/v3/projects/3caa2ea846f7449b8c483788fc5846c8/users/b22838a8ad3949e59cdbaaa1801cd07f/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:35:24.950964 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e4c961-571f-4f42-8d4f-59f3c0ca72df None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:25.008973 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b3039-00fe-4460-8ecc-6aa256aff38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:25.010068 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b3039-00fe-4460-8ecc-6aa256aff38b None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:25.010550 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b3039-00fe-4460-8ecc-6aa256aff38b None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:25.011045 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b3039-00fe-4460-8ecc-6aa256aff38b None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:25.011887 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090b3039-00fe-4460-8ecc-6aa256aff38b None admin] RBAC: Adding query filter params (name=network) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:25.012401 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090b3039-00fe-4460-8ecc-6aa256aff38b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:25.014087 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090b3039-00fe-4460-8ecc-6aa256aff38b None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:25.024277 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 390/777] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 10 22:35:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:35:25.030476 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ce7ed-0b80-4ab6-9667-2a68d8444705 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:25.154713 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ce7ed-0b80-4ab6-9667-2a68d8444705 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:25.202768 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38a989-ff0b-4b0a-bd7f-d28d43b09792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:25.203445 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38a989-ff0b-4b0a-bd7f-d28d43b09792 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:25.203747 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38a989-ff0b-4b0a-bd7f-d28d43b09792 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:25.204767 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38a989-ff0b-4b0a-bd7f-d28d43b09792 None admin] PATH_INFO: `/v3/services` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:25.205464 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38a989-ff0b-4b0a-bd7f-d28d43b09792 None admin] RBAC: Adding query filter params (type=network) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:25.205821 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38a989-ff0b-4b0a-bd7f-d28d43b09792 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:25.207187 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38a989-ff0b-4b0a-bd7f-d28d43b09792 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:25.208399 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:25.217206 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 388/778] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 10 22:35:25 2022] GET /identity/v3/services?type=network => generated 350 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:25.225170 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b3039-00fe-4460-8ecc-6aa256aff38b None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:25.355108 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b3039-00fe-4460-8ecc-6aa256aff38b None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:25.369787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:25.404701 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a33800-0d90-4cb8-a545-58b00d84bc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:25.405396 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a33800-0d90-4cb8-a545-58b00d84bc2a None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:25.405667 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a33800-0d90-4cb8-a545-58b00d84bc2a None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:25.405885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a33800-0d90-4cb8-a545-58b00d84bc2a None admin] PATH_INFO: `/v3/endpoints` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:25.406646 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a33800-0d90-4cb8-a545-58b00d84bc2a None admin] RBAC: Adding query filter params (interface=public, service_id=078782f9a2604c36b49904382ba10468, region_id=RegionOne) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:25.407504 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a33800-0d90-4cb8-a545-58b00d84bc2a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:25.408895 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a33800-0d90-4cb8-a545-58b00d84bc2a None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:25.420023 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 391/779] 10.222.0.76 () {56 vars in 1251 bytes} [Wed Aug 10 22:35:25 2022] GET /identity/v3/endpoints?service_id=078782f9a2604c36b49904382ba10468&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:25.426186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38a989-ff0b-4b0a-bd7f-d28d43b09792 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:25.494312 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953bdab9-d162-4658-ab7b-5065c836d843 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:25.494755 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953bdab9-d162-4658-ab7b-5065c836d843 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:25.495122 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953bdab9-d162-4658-ab7b-5065c836d843 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:25.496814 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 392/780] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:35:25.502472 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a66619-a70f-401d-89c0-b216a0a3c75b None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:25.502868 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a66619-a70f-401d-89c0-b216a0a3c75b None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:25.503241 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a66619-a70f-401d-89c0-b216a0a3c75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:25.537244 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-22a66619-a70f-401d-89c0-b216a0a3c75b None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:25.538505 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-22a66619-a70f-401d-89c0-b216a0a3c75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:25.603015 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38a989-ff0b-4b0a-bd7f-d28d43b09792 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:25.664104 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86805c09-d14d-4325-b4fe-a25018a60b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:25.664642 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86805c09-d14d-4325-b4fe-a25018a60b43 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:25.665600 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86805c09-d14d-4325-b4fe-a25018a60b43 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:25.665880 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86805c09-d14d-4325-b4fe-a25018a60b43 None admin] PATH_INFO: `/v3/services` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:25.666484 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86805c09-d14d-4325-b4fe-a25018a60b43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:25.667848 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86805c09-d14d-4325-b4fe-a25018a60b43 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:25.669783 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 393/781] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:35:25.682092 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 389/782] 10.222.0.76 () {56 vars in 1087 bytes} [Wed Aug 10 22:35:25 2022] GET /identity/v3/services => generated 772 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:25.698612 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a66619-a70f-401d-89c0-b216a0a3c75b None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:25.808426 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a66619-a70f-401d-89c0-b216a0a3c75b None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:25.817147 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:25.845742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cab013-3e87-4e81-a730-e9370bd35933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:25.846387 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cab013-3e87-4e81-a730-e9370bd35933 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:25.846663 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cab013-3e87-4e81-a730-e9370bd35933 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:25.846932 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cab013-3e87-4e81-a730-e9370bd35933 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:25.858464 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cab013-3e87-4e81-a730-e9370bd35933 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:35:25.863978 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-61cab013-3e87-4e81-a730-e9370bd35933 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:35:25.863978 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:25.863978 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:25.863978 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:25.863978 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:25.863978 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:25.863978 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:25.863978 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:25.863978 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:25.863978 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:25.863978 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:25.863978 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:25.863978 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:35:25.863978 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:35:25.863978 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:35:25.863978 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:35:25.864983 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:35:25.864983 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:35:25.864983 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:35:25.864983 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:35:25.864983 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:25.864983 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:25.864983 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:25.864983 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:25.864983 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:25.864983 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:25.864983 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:25.864983 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:25.864983 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:25.864983 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:25.864983 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:25.864983 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:25.865882 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:25.865882 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:25.865882 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:25.865882 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:25.865882 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:35:25.865882 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:35:25.865882 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:35:25.865882 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:35:25.865882 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:35:25.865882 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:35:25.865882 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:35:25.865882 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:25.869404 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 394/783] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:35:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:25.871610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86805c09-d14d-4325-b4fe-a25018a60b43 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:25.940464 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:25.984958 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86805c09-d14d-4325-b4fe-a25018a60b43 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:26.019822 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f1cde7-48da-4f9b-8e96-bcfd609d7297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:26.020433 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1cde7-48da-4f9b-8e96-bcfd609d7297 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:26.020713 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1cde7-48da-4f9b-8e96-bcfd609d7297 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:26.020982 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1cde7-48da-4f9b-8e96-bcfd609d7297 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:26.021660 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f1cde7-48da-4f9b-8e96-bcfd609d7297 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:26.021956 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f1cde7-48da-4f9b-8e96-bcfd609d7297 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:26.023306 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f1cde7-48da-4f9b-8e96-bcfd609d7297 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:26.036735 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 390/784] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:35:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:26.038093 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cab013-3e87-4e81-a730-e9370bd35933 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:26.139772 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cab013-3e87-4e81-a730-e9370bd35933 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:26.174426 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5a408-3dee-4ee9-80f7-bf7d21ce9c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:26.175055 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5a408-3dee-4ee9-80f7-bf7d21ce9c94 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:26.175333 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5a408-3dee-4ee9-80f7-bf7d21ce9c94 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:26.175602 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5a408-3dee-4ee9-80f7-bf7d21ce9c94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:26.185889 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c5a408-3dee-4ee9-80f7-bf7d21ce9c94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:26.187392 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c5a408-3dee-4ee9-80f7-bf7d21ce9c94 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:26.202770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-34c5a408-3dee-4ee9-80f7-bf7d21ce9c94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:26.202770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:26.202770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:26.202770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:26.202770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:26.202770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:26.202770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:26.202770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:26.202770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:26.202770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:26.202770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:26.202770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:26.202770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:26.202770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:26.202770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:26.202770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:26.204467 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:26.204467 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:26.204467 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:26.204467 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:26.204467 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:26.204467 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:26.204467 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:26.204467 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:26.204467 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:26.204467 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:26.204467 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:26.204467 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:26.204467 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:26.204467 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:26.204467 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:26.204467 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:26.204467 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:26.206303 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:26.206303 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:26.206303 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:26.211264 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 395/785] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:26.211452 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f1cde7-48da-4f9b-8e96-bcfd609d7297 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:26.314091 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f1cde7-48da-4f9b-8e96-bcfd609d7297 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:26.339824 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:26.358917 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e48eb-f8bc-4c89-9314-1a68e7734aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:26.359677 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e48eb-f8bc-4c89-9314-1a68e7734aff None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:26.360033 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e48eb-f8bc-4c89-9314-1a68e7734aff None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:26.360307 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e48eb-f8bc-4c89-9314-1a68e7734aff None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:26.360961 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e48eb-f8bc-4c89-9314-1a68e7734aff None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:26.361260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e48eb-f8bc-4c89-9314-1a68e7734aff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:26.362648 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e48eb-f8bc-4c89-9314-1a68e7734aff None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:26.381430 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 391/786] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:26.383135 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5a408-3dee-4ee9-80f7-bf7d21ce9c94 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:26.410631 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:26.504745 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5a408-3dee-4ee9-80f7-bf7d21ce9c94 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:26.539684 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81698136-7cb4-409c-8d30-083e9c3c78a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:26.540323 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81698136-7cb4-409c-8d30-083e9c3c78a6 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:26.540582 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81698136-7cb4-409c-8d30-083e9c3c78a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:26.540850 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81698136-7cb4-409c-8d30-083e9c3c78a6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:26.549846 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81698136-7cb4-409c-8d30-083e9c3c78a6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:26.551305 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81698136-7cb4-409c-8d30-083e9c3c78a6 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:26.562628 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-81698136-7cb4-409c-8d30-083e9c3c78a6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 22:35:26.562628 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:26.562628 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:26.562628 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:26.562628 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:26.562628 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:26.562628 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:26.562628 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:26.562628 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:26.562628 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:26.562628 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:26.562628 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:26.562628 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:35:26.562628 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:35:26.562628 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:35:26.562628 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:35:26.564617 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:26.564617 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:26.564617 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:35:26.564617 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:26.564617 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:35:26.564617 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:26.564617 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:26.564617 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:26.564617 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:26.564617 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:26.564617 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:26.564617 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:26.564617 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:26.564617 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:26.564617 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:26.564617 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:26.564617 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:26.564617 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:26.565634 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:26.565634 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:26.565634 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:35:26.565634 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:35:26.565634 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:35:26.565634 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:35:26.565634 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:35:26.565634 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:35:26.565634 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 22:35:26.565634 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:26.567939 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 396/787] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:35:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:26.570690 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e48eb-f8bc-4c89-9314-1a68e7734aff None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:26.675159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e48eb-f8bc-4c89-9314-1a68e7734aff None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:26.712885 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6b64c5-abf6-4a9c-8eb1-6577ebb456ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:26.713578 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6b64c5-abf6-4a9c-8eb1-6577ebb456ae None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:26.713851 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6b64c5-abf6-4a9c-8eb1-6577ebb456ae None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:26.714117 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6b64c5-abf6-4a9c-8eb1-6577ebb456ae None admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:26.714937 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6b64c5-abf6-4a9c-8eb1-6577ebb456ae None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:26.715239 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6b64c5-abf6-4a9c-8eb1-6577ebb456ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:26.717233 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6b64c5-abf6-4a9c-8eb1-6577ebb456ae None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:26.741521 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 392/788] 10.222.0.76 () {56 vars in 1135 bytes} [Wed Aug 10 22:35:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:26.743988 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81698136-7cb4-409c-8d30-083e9c3c78a6 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:26.828937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:26.852766 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81698136-7cb4-409c-8d30-083e9c3c78a6 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:26.888752 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3c464-21a3-42cf-92be-392328d4ab46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:26.889415 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3c464-21a3-42cf-92be-392328d4ab46 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:26.889691 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3c464-21a3-42cf-92be-392328d4ab46 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:26.889958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3c464-21a3-42cf-92be-392328d4ab46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:26.901714 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3c464-21a3-42cf-92be-392328d4ab46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:26.903155 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3c464-21a3-42cf-92be-392328d4ab46 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:26.918009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-01c3c464-21a3-42cf-92be-392328d4ab46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:26.918009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:26.918009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:26.918009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:26.918009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:26.918009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:26.918009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:26.918009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:26.918009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:26.918009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:26.918009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:26.918009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:26.918009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:26.918009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:26.918009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:26.918009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:26.920317 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:26.920317 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:26.920317 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:26.920317 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:26.920317 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:26.920317 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:26.920317 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:26.920317 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:26.920317 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:26.920317 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:26.920317 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:26.920317 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:26.920317 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:26.920317 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:26.920317 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:26.920317 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:26.920317 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:26.922486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:26.922486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:26.922486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:26.924304 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 397/789] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:26.925329 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6b64c5-abf6-4a9c-8eb1-6577ebb456ae None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:26.972610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:27.028046 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e918095-6e11-4e93-8a49-8da1bc610fa0 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:27.028384 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e918095-6e11-4e93-8a49-8da1bc610fa0 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:27.028590 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e918095-6e11-4e93-8a49-8da1bc610fa0 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:27.030212 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 398/790] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:27.037699 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6b64c5-abf6-4a9c-8eb1-6577ebb456ae None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:27.041296 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7627ac-fbab-438e-8dbe-d3f655a22bcc None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:27.041589 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7627ac-fbab-438e-8dbe-d3f655a22bcc None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:27.041859 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7627ac-fbab-438e-8dbe-d3f655a22bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:27.076171 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebedf903-4117-4fe6-bd3f-01a39ed9b56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:27.076778 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebedf903-4117-4fe6-bd3f-01a39ed9b56c None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:27.077109 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebedf903-4117-4fe6-bd3f-01a39ed9b56c None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:27.077411 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebedf903-4117-4fe6-bd3f-01a39ed9b56c None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:27.077700 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-8d7627ac-fbab-438e-8dbe-d3f655a22bcc None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:27.078048 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebedf903-4117-4fe6-bd3f-01a39ed9b56c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:27.079173 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebedf903-4117-4fe6-bd3f-01a39ed9b56c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:27.079832 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-8d7627ac-fbab-438e-8dbe-d3f655a22bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:27.080484 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebedf903-4117-4fe6-bd3f-01a39ed9b56c None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:27.092607 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 393/791] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:27.099541 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594dc283-a90d-477c-b5fe-7d12a579a58d None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:27.099822 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594dc283-a90d-477c-b5fe-7d12a579a58d None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:27.100091 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594dc283-a90d-477c-b5fe-7d12a579a58d None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:27.102161 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 394/792] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:27 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:27.102848 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594dc283-a90d-477c-b5fe-7d12a579a58d None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:27.169871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 399/793] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:35:27.171248 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8887e9-f373-4d6c-8def-81d8a1ea4447 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:27.171528 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8887e9-f373-4d6c-8def-81d8a1ea4447 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:27.171802 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8887e9-f373-4d6c-8def-81d8a1ea4447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:27.201567 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-3c8887e9-f373-4d6c-8def-81d8a1ea4447 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:27.203009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-3c8887e9-f373-4d6c-8def-81d8a1ea4447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:27.240586 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594dc283-a90d-477c-b5fe-7d12a579a58d None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:27.242786 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:27.269752 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:27.283842 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 400/794] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:35:27.284919 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8887e9-f373-4d6c-8def-81d8a1ea4447 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:27.287617 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c1ddd-f6eb-46e6-9e55-735f23a9acb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:27.289331 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c1ddd-f6eb-46e6-9e55-735f23a9acb8 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:27.289717 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c1ddd-f6eb-46e6-9e55-735f23a9acb8 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:27.290089 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c1ddd-f6eb-46e6-9e55-735f23a9acb8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:27.303679 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927c1ddd-f6eb-46e6-9e55-735f23a9acb8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:27.303679 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927c1ddd-f6eb-46e6-9e55-735f23a9acb8 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:27.318009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-927c1ddd-f6eb-46e6-9e55-735f23a9acb8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:27.318009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:27.318009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:27.318009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:27.318009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:27.318009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:27.318009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:27.318009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:27.318009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:27.318009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:27.318009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:27.318009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:27.318009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:35:27.318009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:35:27.318009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:35:27.319159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:35:27.319159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:27.319159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:27.319159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:27.319159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:27.319159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:27.319159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:27.319159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:27.319159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:27.319159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:27.319159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:27.319159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:27.319159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:27.319159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:27.319159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:27.319159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:27.319159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:27.320278 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:35:27.320278 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:35:27.320278 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:35:27.320278 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:35:27.320278 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:35:27.320278 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:35:27.320278 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:27.320278 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:27.321768 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 395/795] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:35:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:27.322616 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c1ddd-f6eb-46e6-9e55-735f23a9acb8 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:27.444211 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8887e9-f373-4d6c-8def-81d8a1ea4447 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:27.448008 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c1ddd-f6eb-46e6-9e55-735f23a9acb8 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:27.485411 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1170fc-c5b5-4456-a547-dfc348088522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:27.486477 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1170fc-c5b5-4456-a547-dfc348088522 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:27.487104 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1170fc-c5b5-4456-a547-dfc348088522 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:27.487616 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1170fc-c5b5-4456-a547-dfc348088522 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:27.492062 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344547d-d639-41fd-ba55-a0be576b64d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:27.492765 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344547d-d639-41fd-ba55-a0be576b64d3 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:27.493049 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344547d-d639-41fd-ba55-a0be576b64d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:27.493345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344547d-d639-41fd-ba55-a0be576b64d3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:27.494263 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8344547d-d639-41fd-ba55-a0be576b64d3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:27.495651 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8344547d-d639-41fd-ba55-a0be576b64d3 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:27.502936 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1170fc-c5b5-4456-a547-dfc348088522 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:35:27.508088 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-0e1170fc-c5b5-4456-a547-dfc348088522 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:35:27.508088 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:27.508088 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:27.508088 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:27.508088 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:27.508088 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:27.508088 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:27.508088 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:27.508088 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:27.508088 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:27.508088 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:27.508088 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:27.508088 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:35:27.508088 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:35:27.508088 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:35:27.508088 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:35:27.509037 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:35:27.509037 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:35:27.509037 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:35:27.509037 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:35:27.509037 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:27.509037 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:27.509037 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:27.509037 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:27.509037 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:27.509037 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:27.509037 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:27.509037 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:27.509037 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:27.509037 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:27.509037 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:27.509037 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:27.509905 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:27.509905 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:27.509905 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:27.509905 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:27.509905 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:35:27.509905 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:35:27.509905 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:35:27.509905 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:35:27.509905 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:35:27.509905 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:35:27.509905 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:35:27.509905 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:27.513817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-8344547d-d639-41fd-ba55-a0be576b64d3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 22:35:27.513817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:27.513817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:35:27.513817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:35:27.513817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:35:27.513817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:35:27.513817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:35:27.513817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:27.513817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 22:35:27.513817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:27.513817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:35:27.513817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:27.513817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:27.513817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:27.513817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:27.513817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:27.513817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:27.514775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:27.514775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:27.514775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:27.514775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:27.514775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:27.514775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:27.514775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:35:27.514775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:35:27.514775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:35:27.514775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:35:27.514775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:27.514775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:27.514775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:27.514775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:27.514775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:27.514775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:27.515597 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:27.515597 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:27.515597 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:27.515597 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:27.515597 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:27.515597 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:27.515597 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:27.515597 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:27.515597 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:27.515597 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:27.515597 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:35:27.515597 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:27.515597 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 22:35:27.515597 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:27.517257 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 396/796] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 10 22:35:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:27.517257 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1170fc-c5b5-4456-a547-dfc348088522 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:27.520792 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 401/797] 10.222.0.76 () {56 vars in 1129 bytes} [Wed Aug 10 22:35:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:27.521872 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344547d-d639-41fd-ba55-a0be576b64d3 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:27.554068 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:27.585610 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:27.642722 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344547d-d639-41fd-ba55-a0be576b64d3 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:27.651248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1170fc-c5b5-4456-a547-dfc348088522 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:27.689487 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2dc942-88eb-4d1f-988a-dcd3cbd84ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:27.690114 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2dc942-88eb-4d1f-988a-dcd3cbd84ee9 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:27.690283 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2dc942-88eb-4d1f-988a-dcd3cbd84ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:27.690623 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2dc942-88eb-4d1f-988a-dcd3cbd84ee9 None admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:27.691252 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2dc942-88eb-4d1f-988a-dcd3cbd84ee9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:27.691557 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2dc942-88eb-4d1f-988a-dcd3cbd84ee9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:27.691784 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461be7de-801f-4219-ad0f-388c7caedfbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:27.692404 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461be7de-801f-4219-ad0f-388c7caedfbb None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:27.692687 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461be7de-801f-4219-ad0f-388c7caedfbb None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:27.692961 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461be7de-801f-4219-ad0f-388c7caedfbb None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:27.693068 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2dc942-88eb-4d1f-988a-dcd3cbd84ee9 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:27.693661 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461be7de-801f-4219-ad0f-388c7caedfbb None admin] RBAC: Adding query filter params (name=service) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:27.693967 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461be7de-801f-4219-ad0f-388c7caedfbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:27.695525 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461be7de-801f-4219-ad0f-388c7caedfbb None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:27.713456 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 402/798] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 10 22:35:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:27.713660 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 397/799] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:35:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:27.713854 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461be7de-801f-4219-ad0f-388c7caedfbb None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:27.713939 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb322ef-2f7e-4c80-96e5-1deede6352de None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:27.713939 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb322ef-2f7e-4c80-96e5-1deede6352de None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:27.713939 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb322ef-2f7e-4c80-96e5-1deede6352de None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:27.713939 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 398/800] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:27.714187 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb322ef-2f7e-4c80-96e5-1deede6352de None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:27.811197 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:27.828346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461be7de-801f-4219-ad0f-388c7caedfbb None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:27.833806 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb322ef-2f7e-4c80-96e5-1deede6352de None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:27.846305 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:27.869742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fff0c9-7e00-4214-8930-08488007e0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:27.870731 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fff0c9-7e00-4214-8930-08488007e0d5 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:27.871188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fff0c9-7e00-4214-8930-08488007e0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:27.871631 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fff0c9-7e00-4214-8930-08488007e0d5 None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:27.872622 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fff0c9-7e00-4214-8930-08488007e0d5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:27.873097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fff0c9-7e00-4214-8930-08488007e0d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:27.873506 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a4cf03-49ec-4b9d-a451-2e832f701235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:27.874174 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4cf03-49ec-4b9d-a451-2e832f701235 None admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:27.874455 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4cf03-49ec-4b9d-a451-2e832f701235 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:27.874732 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4cf03-49ec-4b9d-a451-2e832f701235 None admin] PATH_INFO: `/v3/projects/3caa2ea846f7449b8c483788fc5846c8/users/1e3a76df95a34b9fb9816f22b85d1071/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:27.875545 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fff0c9-7e00-4214-8930-08488007e0d5 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:27.876951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a4cf03-49ec-4b9d-a451-2e832f701235 None admin] RBAC: Authorizing `identity:create_grant(project_id=3caa2ea846f7449b8c483788fc5846c8, user_id=1e3a76df95a34b9fb9816f22b85d1071, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:27.878419 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a4cf03-49ec-4b9d-a451-2e832f701235 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:27.884836 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 403/801] 10.222.0.76 () {56 vars in 1124 bytes} [Wed Aug 10 22:35:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:27.884836 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fff0c9-7e00-4214-8930-08488007e0d5 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:27.905330 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 399/802] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 10 22:35:27 2022] PUT /identity/v3/projects/3caa2ea846f7449b8c483788fc5846c8/users/1e3a76df95a34b9fb9816f22b85d1071/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:35:27.905330 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9eb9b-002a-40f9-8d94-4afa54aeb9aa None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:27.905330 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9eb9b-002a-40f9-8d94-4afa54aeb9aa None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:27.905330 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9eb9b-002a-40f9-8d94-4afa54aeb9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:27.932377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-f4c9eb9b-002a-40f9-8d94-4afa54aeb9aa None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:27.934624 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-f4c9eb9b-002a-40f9-8d94-4afa54aeb9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:28.018118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fff0c9-7e00-4214-8930-08488007e0d5 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:28.050284 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 400/803] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:35:28.051629 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9eb9b-002a-40f9-8d94-4afa54aeb9aa None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:28.062049 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da4c6c1-b71e-4f59-ac3b-72833d30913f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:28.062835 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4c6c1-b71e-4f59-ac3b-72833d30913f None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:28.063176 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4c6c1-b71e-4f59-ac3b-72833d30913f None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:28.063520 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4c6c1-b71e-4f59-ac3b-72833d30913f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:28.076143 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da4c6c1-b71e-4f59-ac3b-72833d30913f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:28.077768 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da4c6c1-b71e-4f59-ac3b-72833d30913f None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:28.093593 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-1da4c6c1-b71e-4f59-ac3b-72833d30913f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:28.093593 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:28.093593 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:28.093593 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:28.093593 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:28.093593 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:28.093593 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:28.093593 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:28.093593 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:28.093593 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:28.093593 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:28.093593 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:28.093593 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:28.093593 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:28.093593 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:28.093593 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:28.095116 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:28.095116 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:28.095116 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:28.095116 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:28.095116 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:28.095116 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:28.095116 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:28.095116 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:28.095116 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:28.095116 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:28.095116 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:28.095116 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:28.095116 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:28.095116 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:28.095116 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:28.095116 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:28.095116 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:28.095998 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:28.095998 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:28.095998 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:28.095998 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 404/804] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:28.096816 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da4c6c1-b71e-4f59-ac3b-72833d30913f None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:28.149684 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:28.156715 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:28.189076 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9eb9b-002a-40f9-8d94-4afa54aeb9aa None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:28.225795 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da4c6c1-b71e-4f59-ac3b-72833d30913f None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:28.226141 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9151dd-ccd7-409b-842e-aecbdbcef937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:28.226912 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9151dd-ccd7-409b-842e-aecbdbcef937 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:28.227409 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9151dd-ccd7-409b-842e-aecbdbcef937 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:28.227870 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9151dd-ccd7-409b-842e-aecbdbcef937 None admin] PATH_INFO: `/v3/services` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:28.229203 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9151dd-ccd7-409b-842e-aecbdbcef937 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:28.230465 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9151dd-ccd7-409b-842e-aecbdbcef937 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:28.231869 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9151dd-ccd7-409b-842e-aecbdbcef937 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:28.239375 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 401/805] 10.222.0.76 () {56 vars in 1124 bytes} [Wed Aug 10 22:35:28 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:28.244182 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9151dd-ccd7-409b-842e-aecbdbcef937 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:28.270536 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955bc7d4-b99d-4e72-9e6e-2ab39faf7d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:28.271601 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bc7d4-b99d-4e72-9e6e-2ab39faf7d2b None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:28.272099 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bc7d4-b99d-4e72-9e6e-2ab39faf7d2b None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:28.272593 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bc7d4-b99d-4e72-9e6e-2ab39faf7d2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:28.284411 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955bc7d4-b99d-4e72-9e6e-2ab39faf7d2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:35:28.289769 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-955bc7d4-b99d-4e72-9e6e-2ab39faf7d2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:35:28.289769 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:28.289769 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:28.289769 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:28.289769 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:28.289769 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:28.289769 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:28.289769 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:28.289769 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:28.289769 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:28.289769 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:28.289769 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:28.289769 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:35:28.289769 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:35:28.289769 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:35:28.289769 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:35:28.290730 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:35:28.290730 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:35:28.290730 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:35:28.290730 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:35:28.290730 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:28.290730 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:28.290730 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:28.290730 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:28.290730 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:28.290730 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:28.290730 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:28.290730 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:28.290730 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:28.290730 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:28.290730 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:28.290730 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:28.291958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:28.291958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:28.291958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:28.291958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:28.291958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:35:28.291958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:35:28.291958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:35:28.291958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:35:28.291958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:35:28.291958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:35:28.291958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:35:28.291958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:28.291958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 405/806] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:35:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:28.292659 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955bc7d4-b99d-4e72-9e6e-2ab39faf7d2b None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:28.354662 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9151dd-ccd7-409b-842e-aecbdbcef937 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:28.392481 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a230ec9-3ee3-444e-8ffd-0f594a6f8ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:28.393434 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a230ec9-3ee3-444e-8ffd-0f594a6f8ee3 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:28.394115 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a230ec9-3ee3-444e-8ffd-0f594a6f8ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:28.394390 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a230ec9-3ee3-444e-8ffd-0f594a6f8ee3 None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:28.395030 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a230ec9-3ee3-444e-8ffd-0f594a6f8ee3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:28.395328 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a230ec9-3ee3-444e-8ffd-0f594a6f8ee3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:28.396798 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a230ec9-3ee3-444e-8ffd-0f594a6f8ee3 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:28.397585 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:28.400313 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:28.408692 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955bc7d4-b99d-4e72-9e6e-2ab39faf7d2b None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:28.411268 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 402/807] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:28.412498 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b0225-02ec-426c-af17-97f91193e684 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:28.412783 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b0225-02ec-426c-af17-97f91193e684 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:28.413057 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b0225-02ec-426c-af17-97f91193e684 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:28.414446 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 403/808] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:28.414970 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b0225-02ec-426c-af17-97f91193e684 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:28.457065 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebe0605-8049-4607-ad95-d247b4df1830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:28.457985 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe0605-8049-4607-ad95-d247b4df1830 None admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:28.458267 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe0605-8049-4607-ad95-d247b4df1830 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:28.458538 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe0605-8049-4607-ad95-d247b4df1830 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:28.459197 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebe0605-8049-4607-ad95-d247b4df1830 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:28.460555 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebe0605-8049-4607-ad95-d247b4df1830 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:28.493324 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 406/809] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:35:28 2022] POST /identity/v3/endpoints => generated 346 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:35:28.493324 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebe0605-8049-4607-ad95-d247b4df1830 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:28.551905 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b0225-02ec-426c-af17-97f91193e684 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:28.590772 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b161761-9d5f-4acf-a1c1-b763a52d9cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:28.591426 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b161761-9d5f-4acf-a1c1-b763a52d9cc1 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:28.591699 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b161761-9d5f-4acf-a1c1-b763a52d9cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:28.591966 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b161761-9d5f-4acf-a1c1-b763a52d9cc1 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:28.592704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b161761-9d5f-4acf-a1c1-b763a52d9cc1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:28.593008 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b161761-9d5f-4acf-a1c1-b763a52d9cc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:28.595108 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b161761-9d5f-4acf-a1c1-b763a52d9cc1 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:28.611906 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 404/810] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:35:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:28.611906 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281ba68-2b64-4677-8586-304f80581728 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:28.611906 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281ba68-2b64-4677-8586-304f80581728 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:28.611906 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281ba68-2b64-4677-8586-304f80581728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:28.640668 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-7281ba68-2b64-4677-8586-304f80581728 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:28.643161 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-7281ba68-2b64-4677-8586-304f80581728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:28.649515 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebe0605-8049-4607-ad95-d247b4df1830 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:28.705890 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c1c34-c4c6-4381-9f91-67c12e6eb078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:28.706384 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c1c34-c4c6-4381-9f91-67c12e6eb078 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:28.706562 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c1c34-c4c6-4381-9f91-67c12e6eb078 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:28.706854 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c1c34-c4c6-4381-9f91-67c12e6eb078 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:28.715160 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:28.715652 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288c1c34-c4c6-4381-9f91-67c12e6eb078 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:28.717118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288c1c34-c4c6-4381-9f91-67c12e6eb078 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:28.731915 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-288c1c34-c4c6-4381-9f91-67c12e6eb078 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 22:35:28.731915 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:28.731915 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:28.731915 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:28.731915 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:28.731915 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:28.731915 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:28.731915 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:28.731915 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:28.731915 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:28.731915 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:28.731915 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:28.731915 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:35:28.731915 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:35:28.731915 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:35:28.731915 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:35:28.732905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:28.732905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:28.732905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:35:28.732905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:28.732905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:35:28.732905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:28.732905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:28.732905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:28.732905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:28.732905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:28.732905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:28.732905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:28.732905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:28.732905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:28.732905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:28.732905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:28.732905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:28.732905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:28.734021 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:28.734021 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:28.734021 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:35:28.734021 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:35:28.734021 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:35:28.734021 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:35:28.734021 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:35:28.734021 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:35:28.734021 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 22:35:28.734021 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:28.734021 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 407/811] 10.222.0.76 () {56 vars in 1117 bytes} [Wed Aug 10 22:35:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:28.734021 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c1c34-c4c6-4381-9f91-67c12e6eb078 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:28.746958 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 405/812] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:35:28.747995 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7281ba68-2b64-4677-8586-304f80581728 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:28.749839 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:28.848894 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c1c34-c4c6-4381-9f91-67c12e6eb078 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:28.884928 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efabda-5c06-4155-b00b-e4610436d20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:28.885687 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efabda-5c06-4155-b00b-e4610436d20a None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:28.886041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efabda-5c06-4155-b00b-e4610436d20a None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:28.886394 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efabda-5c06-4155-b00b-e4610436d20a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:28.895286 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7281ba68-2b64-4677-8586-304f80581728 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:28.900203 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15efabda-5c06-4155-b00b-e4610436d20a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:28.901701 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15efabda-5c06-4155-b00b-e4610436d20a None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:28.920013 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-15efabda-5c06-4155-b00b-e4610436d20a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:28.920013 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:28.920013 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:28.920013 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:28.920013 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:28.920013 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:28.920013 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:28.920013 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:28.920013 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:28.920013 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:28.920013 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:28.920013 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:28.920013 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:28.920013 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:28.920013 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:28.920013 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:28.921060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:28.921060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:28.921060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:28.921060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:28.921060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:28.921060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:28.921060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:28.921060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:28.921060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:28.921060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:28.921060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:28.921060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:28.921060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:28.921060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:28.921060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:28.921060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:28.921060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:28.923023 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:28.923023 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:28.923023 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:28.923023 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 408/813] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:28.923023 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efabda-5c06-4155-b00b-e4610436d20a None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:28.948076 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637f082-5179-4da4-ae49-eb6228a84a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:28.949044 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637f082-5179-4da4-ae49-eb6228a84a2d None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:28.949986 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637f082-5179-4da4-ae49-eb6228a84a2d None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:28.950506 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637f082-5179-4da4-ae49-eb6228a84a2d None admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:28.951732 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8637f082-5179-4da4-ae49-eb6228a84a2d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:28.952258 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8637f082-5179-4da4-ae49-eb6228a84a2d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:28.955282 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8637f082-5179-4da4-ae49-eb6228a84a2d None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:28.978892 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 406/814] 10.222.0.76 () {56 vars in 1149 bytes} [Wed Aug 10 22:35:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:28.980125 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637f082-5179-4da4-ae49-eb6228a84a2d None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:28.992602 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:29.041799 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efabda-5c06-4155-b00b-e4610436d20a None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:29.068912 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:29.079642 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d44686-c81f-4eaa-9104-00eb2483d362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:29.080538 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d44686-c81f-4eaa-9104-00eb2483d362 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:29.081040 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d44686-c81f-4eaa-9104-00eb2483d362 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:29.081554 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d44686-c81f-4eaa-9104-00eb2483d362 None admin] PATH_INFO: `/v3/services/network` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:29.082347 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d44686-c81f-4eaa-9104-00eb2483d362 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:29.083900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d44686-c81f-4eaa-9104-00eb2483d362 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:29.092816 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-63d44686-c81f-4eaa-9104-00eb2483d362 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 22:35:29.092816 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:29.092816 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:35:29.092816 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:35:29.092816 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:35:29.092816 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:35:29.092816 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:35:29.092816 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:29.092816 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 22:35:29.092816 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:29.092816 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:35:29.092816 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:29.092816 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:29.092816 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:29.092816 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:29.092816 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:29.092816 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:29.094428 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:29.094428 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:29.094428 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:29.094428 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:29.094428 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:29.094428 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:29.094428 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:35:29.094428 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:35:29.094428 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:35:29.094428 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:35:29.094428 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:29.094428 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:29.094428 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:29.094428 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:29.094428 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:29.094428 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:29.095253 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:29.095253 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:29.095253 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:29.095253 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:29.095253 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:29.095253 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:29.095253 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:29.095253 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:29.095253 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:29.095253 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:29.095253 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:35:29.095253 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:29.095253 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 22:35:29.095253 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:29.101765 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 409/815] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:35:28 2022] GET /identity/v3/services/network => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:29.102861 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d44686-c81f-4eaa-9104-00eb2483d362 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:29.108512 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637f082-5179-4da4-ae49-eb6228a84a2d None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:29.160808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef51f54-6f71-4487-a988-b61a82e95d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:29.161462 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef51f54-6f71-4487-a988-b61a82e95d59 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:29.161745 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef51f54-6f71-4487-a988-b61a82e95d59 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:29.162023 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef51f54-6f71-4487-a988-b61a82e95d59 None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:29.162666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef51f54-6f71-4487-a988-b61a82e95d59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:29.162992 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef51f54-6f71-4487-a988-b61a82e95d59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:29.165099 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef51f54-6f71-4487-a988-b61a82e95d59 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:29.181729 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 407/816] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:29.182675 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef51f54-6f71-4487-a988-b61a82e95d59 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:29.213546 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d44686-c81f-4eaa-9104-00eb2483d362 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:29.252784 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6e3b42-d6cf-470b-b3f9-7b412b73dc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:29.254282 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e3b42-d6cf-470b-b3f9-7b412b73dc09 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:29.255044 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e3b42-d6cf-470b-b3f9-7b412b73dc09 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:29.255571 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e3b42-d6cf-470b-b3f9-7b412b73dc09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:29.269507 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:29.270515 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6e3b42-d6cf-470b-b3f9-7b412b73dc09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:29.272535 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6e3b42-d6cf-470b-b3f9-7b412b73dc09 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:29.291016 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-ee6e3b42-d6cf-470b-b3f9-7b412b73dc09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:29.291016 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:29.291016 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:29.291016 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:29.291016 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:29.291016 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:29.291016 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:29.291016 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:29.291016 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:29.291016 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:29.291016 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:29.291016 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:29.291016 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:29.291016 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:29.291016 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:29.291016 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:29.292011 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:29.292011 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:29.292011 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:29.292011 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:29.292011 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:29.292011 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:29.292011 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:29.292011 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:29.292011 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:29.292011 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:29.292011 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:29.292011 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:29.292011 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:29.292011 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:29.292011 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:29.292011 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:29.292011 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:29.292874 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:29.292874 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:29.292874 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:29.299292 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 410/817] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:29.299292 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6e3b42-d6cf-470b-b3f9-7b412b73dc09 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:29.316730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef51f54-6f71-4487-a988-b61a82e95d59 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:29.319110 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:29.361703 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ddb0ee-e05c-4624-bcb2-9fb873a25f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:29.362235 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ddb0ee-e05c-4624-bcb2-9fb873a25f88 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:29.362516 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ddb0ee-e05c-4624-bcb2-9fb873a25f88 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:29.362793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ddb0ee-e05c-4624-bcb2-9fb873a25f88 None admin] PATH_INFO: `/v3/services` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:29.363426 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ddb0ee-e05c-4624-bcb2-9fb873a25f88 None admin] RBAC: Adding query filter params (name=network) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:29.363734 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ddb0ee-e05c-4624-bcb2-9fb873a25f88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:29.365667 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ddb0ee-e05c-4624-bcb2-9fb873a25f88 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:29.376285 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 408/818] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 10 22:35:29 2022] GET /identity/v3/services?name=network => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:29.376285 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ddb0ee-e05c-4624-bcb2-9fb873a25f88 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:29.449256 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6e3b42-d6cf-470b-b3f9-7b412b73dc09 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:29.496806 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9879a-4019-47ae-8e81-91ba15ca1cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:29.498297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9879a-4019-47ae-8e81-91ba15ca1cb2 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:29.498817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9879a-4019-47ae-8e81-91ba15ca1cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:29.499236 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9879a-4019-47ae-8e81-91ba15ca1cb2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:29.503785 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ddb0ee-e05c-4624-bcb2-9fb873a25f88 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:29.510686 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e9879a-4019-47ae-8e81-91ba15ca1cb2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:29.512821 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e9879a-4019-47ae-8e81-91ba15ca1cb2 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:29.529514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-c3e9879a-4019-47ae-8e81-91ba15ca1cb2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:35:29.529514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:29.529514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:29.529514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:29.529514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:29.529514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:29.529514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:29.529514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:29.529514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:29.529514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:29.529514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:29.529514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:29.529514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:35:29.529514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:35:29.529514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:35:29.529514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:35:29.530732 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:29.530732 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:29.530732 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:35:29.530732 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:29.530732 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:35:29.530732 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:29.530732 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:29.530732 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:29.530732 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:29.530732 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:29.530732 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:29.530732 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:29.530732 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:29.530732 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:29.530732 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:29.530732 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:29.530732 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:29.530732 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:29.532313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:29.532313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:29.532313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:35:29.532313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:35:29.532313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:35:29.532313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:35:29.532313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:35:29.532313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:35:29.532313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:35:29.532313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:29.535030 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 411/819] 10.222.0.76 () {56 vars in 1093 bytes} [Wed Aug 10 22:35:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:29.536560 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9879a-4019-47ae-8e81-91ba15ca1cb2 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:29.543962 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5dfdc-6042-4451-a2b9-12ca2e4c0b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:29.544732 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5dfdc-6042-4451-a2b9-12ca2e4c0b77 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:29.545117 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5dfdc-6042-4451-a2b9-12ca2e4c0b77 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:29.545527 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5dfdc-6042-4451-a2b9-12ca2e4c0b77 None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:29.546304 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5dfdc-6042-4451-a2b9-12ca2e4c0b77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:29.546766 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5dfdc-6042-4451-a2b9-12ca2e4c0b77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:29.548400 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5dfdc-6042-4451-a2b9-12ca2e4c0b77 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:29.566429 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 409/820] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:29.566429 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5dfdc-6042-4451-a2b9-12ca2e4c0b77 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:29.591108 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:29.595980 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:29.676282 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5dfdc-6042-4451-a2b9-12ca2e4c0b77 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:29.689263 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9879a-4019-47ae-8e81-91ba15ca1cb2 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:29.715772 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47476ef4-c018-437d-a6c0-a75b5aafd8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:29.717155 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47476ef4-c018-437d-a6c0-a75b5aafd8d7 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:29.717674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47476ef4-c018-437d-a6c0-a75b5aafd8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:29.718200 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47476ef4-c018-437d-a6c0-a75b5aafd8d7 None admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:29.719486 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47476ef4-c018-437d-a6c0-a75b5aafd8d7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:29.720064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47476ef4-c018-437d-a6c0-a75b5aafd8d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:29.721922 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47476ef4-c018-437d-a6c0-a75b5aafd8d7 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:29.732806 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e1927-b302-4623-90d9-a8f3c4ddce6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:29.735167 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e1927-b302-4623-90d9-a8f3c4ddce6d None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:29.735643 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e1927-b302-4623-90d9-a8f3c4ddce6d None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:29.735930 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e1927-b302-4623-90d9-a8f3c4ddce6d None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:29.736582 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e1927-b302-4623-90d9-a8f3c4ddce6d None admin] RBAC: Adding query filter params (type=network) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:29.736885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e1927-b302-4623-90d9-a8f3c4ddce6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:29.738306 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e1927-b302-4623-90d9-a8f3c4ddce6d None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:29.750733 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 412/821] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 10 22:35:29 2022] GET /identity/v3/services?type=network => generated 350 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:29.751048 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 410/822] 10.222.0.76 () {56 vars in 1133 bytes} [Wed Aug 10 22:35:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:29.751737 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e1927-b302-4623-90d9-a8f3c4ddce6d None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:29.752400 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47476ef4-c018-437d-a6c0-a75b5aafd8d7 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:29.861400 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:29.866951 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:29.871590 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e1927-b302-4623-90d9-a8f3c4ddce6d None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:29.884712 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47476ef4-c018-437d-a6c0-a75b5aafd8d7 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:29.909060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14091e0a-d0fe-430e-aaa7-5d0b84d3dd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:29.910275 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14091e0a-d0fe-430e-aaa7-5d0b84d3dd29 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:29.910764 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14091e0a-d0fe-430e-aaa7-5d0b84d3dd29 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:29.911257 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14091e0a-d0fe-430e-aaa7-5d0b84d3dd29 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:29.918605 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14091e0a-d0fe-430e-aaa7-5d0b84d3dd29 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:29.920331 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14091e0a-d0fe-430e-aaa7-5d0b84d3dd29 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:29.925957 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4eaa9-21d3-4370-b2aa-13f249d826dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:29.926622 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4eaa9-21d3-4370-b2aa-13f249d826dc None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:29.926841 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4eaa9-21d3-4370-b2aa-13f249d826dc None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:29.927113 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4eaa9-21d3-4370-b2aa-13f249d826dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:29.933749 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-14091e0a-d0fe-430e-aaa7-5d0b84d3dd29 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:29.933749 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:29.933749 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:29.933749 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:29.933749 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:29.933749 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:29.933749 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:29.933749 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:29.933749 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:29.933749 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:29.933749 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:29.933749 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:29.933749 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:35:29.933749 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:35:29.933749 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:35:29.934626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:35:29.934626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:29.934626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:29.934626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:29.934626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:29.934626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:29.934626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:29.934626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:29.934626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:29.934626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:29.934626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:29.934626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:29.934626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:29.934626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:29.934626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:29.934626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:29.934626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:29.935521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:35:29.935521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:35:29.935521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:35:29.935521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:35:29.935521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:35:29.935521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:35:29.935521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:29.935521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:29.939456 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 413/823] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:35:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:29.940554 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14091e0a-d0fe-430e-aaa7-5d0b84d3dd29 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:30.019329 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda4eaa9-21d3-4370-b2aa-13f249d826dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:30.021175 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda4eaa9-21d3-4370-b2aa-13f249d826dc None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:30.055194 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14091e0a-d0fe-430e-aaa7-5d0b84d3dd29 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:30.092037 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd1f3f8-a20f-4910-bf89-5fcb92393fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:30.092814 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1f3f8-a20f-4910-bf89-5fcb92393fe3 None admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:30.093895 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1f3f8-a20f-4910-bf89-5fcb92393fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:30.094285 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1f3f8-a20f-4910-bf89-5fcb92393fe3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:30.095038 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd1f3f8-a20f-4910-bf89-5fcb92393fe3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:30.096509 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd1f3f8-a20f-4910-bf89-5fcb92393fe3 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:30.118040 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:30.118233 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 414/824] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:35:29 2022] POST /identity/v3/endpoints => generated 337 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:35:30.119562 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd1f3f8-a20f-4910-bf89-5fcb92393fe3 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:30.145568 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:30.158849 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 411/825] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:35:29 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 22:35:30.160461 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168e4e6-04cd-4ad4-81ac-bd337e0a8d3a None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:30.160884 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168e4e6-04cd-4ad4-81ac-bd337e0a8d3a None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:30.161312 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168e4e6-04cd-4ad4-81ac-bd337e0a8d3a None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:30.167107 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 412/826] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:30.167107 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6168e4e6-04cd-4ad4-81ac-bd337e0a8d3a None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:30.299320 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6168e4e6-04cd-4ad4-81ac-bd337e0a8d3a None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:30.300661 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd1f3f8-a20f-4910-bf89-5fcb92393fe3 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:30.340893 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef0647-d4f7-44fc-a80c-1c700994deb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:30.342521 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef0647-d4f7-44fc-a80c-1c700994deb1 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:30.342796 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef0647-d4f7-44fc-a80c-1c700994deb1 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:30.345020 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef0647-d4f7-44fc-a80c-1c700994deb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:30.351405 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b7feb-3f71-4c61-a361-05bf90530fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:30.352325 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b7feb-3f71-4c61-a361-05bf90530fd6 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:30.352705 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b7feb-3f71-4c61-a361-05bf90530fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:30.353097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b7feb-3f71-4c61-a361-05bf90530fd6 None admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:30.353935 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0b7feb-3f71-4c61-a361-05bf90530fd6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:30.354340 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0b7feb-3f71-4c61-a361-05bf90530fd6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:30.355868 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0b7feb-3f71-4c61-a361-05bf90530fd6 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:30.358030 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef0647-d4f7-44fc-a80c-1c700994deb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:30.360077 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef0647-d4f7-44fc-a80c-1c700994deb1 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:30.369846 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 415/827] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:35:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:30.371225 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72a32d-c399-42e4-b939-74d5ca6d749b None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:30.372208 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72a32d-c399-42e4-b939-74d5ca6d749b None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:30.372585 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72a32d-c399-42e4-b939-74d5ca6d749b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:30.377567 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-daef0647-d4f7-44fc-a80c-1c700994deb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:30.377567 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:30.377567 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:30.377567 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:30.377567 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:30.377567 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:30.377567 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:30.377567 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:30.377567 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:30.377567 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:30.377567 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:30.377567 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:30.377567 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:30.377567 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:30.377567 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:30.377567 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:30.378737 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:30.378737 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:30.378737 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:30.378737 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:30.378737 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:30.378737 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:30.378737 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:30.378737 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:30.378737 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:30.378737 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:30.378737 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:30.378737 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:30.378737 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:30.378737 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:30.378737 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:30.378737 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:30.378737 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:30.379710 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:30.379710 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:30.379710 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:30.379710 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 413/828] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:30.379710 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef0647-d4f7-44fc-a80c-1c700994deb1 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:30.399115 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-4e72a32d-c399-42e4-b939-74d5ca6d749b None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:30.400502 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-4e72a32d-c399-42e4-b939-74d5ca6d749b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:30.420444 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:30.480401 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 416/829] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:30 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:35:30.481389 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72a32d-c399-42e4-b939-74d5ca6d749b None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:30.505503 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:30.505768 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef0647-d4f7-44fc-a80c-1c700994deb1 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:30.541836 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec65dbe-6732-4171-8d6e-140efaf25f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:30.542609 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec65dbe-6732-4171-8d6e-140efaf25f88 None admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:30.542988 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec65dbe-6732-4171-8d6e-140efaf25f88 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:30.543322 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec65dbe-6732-4171-8d6e-140efaf25f88 None admin] PATH_INFO: `/v3/projects/3caa2ea846f7449b8c483788fc5846c8/users/aff5e804c64345b298c745287a1d0a32/roles/5dfd0f952cd641bbb2257860c4c892ac` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:30.559181 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec65dbe-6732-4171-8d6e-140efaf25f88 None admin] RBAC: Authorizing `identity:create_grant(project_id=3caa2ea846f7449b8c483788fc5846c8, user_id=aff5e804c64345b298c745287a1d0a32, role_id=5dfd0f952cd641bbb2257860c4c892ac)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:30.560564 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec65dbe-6732-4171-8d6e-140efaf25f88 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:30.574760 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 414/830] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 10 22:35:30 2022] PUT /identity/v3/projects/3caa2ea846f7449b8c483788fc5846c8/users/aff5e804c64345b298c745287a1d0a32/roles/5dfd0f952cd641bbb2257860c4c892ac => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:35:30.575581 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec65dbe-6732-4171-8d6e-140efaf25f88 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:30.608551 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72a32d-c399-42e4-b939-74d5ca6d749b None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:30.653938 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75279c10-a588-4b13-9adf-f6590dac0052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:30.654852 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75279c10-a588-4b13-9adf-f6590dac0052 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:30.655276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75279c10-a588-4b13-9adf-f6590dac0052 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:30.655758 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75279c10-a588-4b13-9adf-f6590dac0052 None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:30.656639 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75279c10-a588-4b13-9adf-f6590dac0052 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:30.657213 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75279c10-a588-4b13-9adf-f6590dac0052 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:30.658884 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75279c10-a588-4b13-9adf-f6590dac0052 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:30.672998 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 417/831] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:30.674470 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f6331-495f-4754-a883-c0b201852a8f None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:30.674808 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f6331-495f-4754-a883-c0b201852a8f None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:30.675086 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f6331-495f-4754-a883-c0b201852a8f None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:30.676562 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 418/832] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:30.678606 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f6331-495f-4754-a883-c0b201852a8f None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:30.699398 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:30.720820 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec65dbe-6732-4171-8d6e-140efaf25f88 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:30.764343 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:30.765838 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af9dba9-aa33-45b0-8f4d-77d696d6796d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:30.766483 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af9dba9-aa33-45b0-8f4d-77d696d6796d None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:30.766765 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af9dba9-aa33-45b0-8f4d-77d696d6796d None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:30.767060 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af9dba9-aa33-45b0-8f4d-77d696d6796d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:30.779911 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af9dba9-aa33-45b0-8f4d-77d696d6796d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:35:30.785739 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-9af9dba9-aa33-45b0-8f4d-77d696d6796d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:35:30.785739 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:30.785739 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:30.785739 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:30.785739 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:30.785739 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:30.785739 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:30.785739 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:30.785739 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:30.785739 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:30.785739 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:30.785739 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:30.785739 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:35:30.785739 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:35:30.785739 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:35:30.785739 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:35:30.786671 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:35:30.786671 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:35:30.786671 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:35:30.786671 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:35:30.786671 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:30.786671 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:30.786671 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:30.786671 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:30.786671 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:30.786671 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:30.786671 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:30.786671 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:30.786671 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:30.786671 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:30.786671 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:30.786671 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:30.788248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:30.788248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:30.788248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:30.788248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:30.788248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:35:30.788248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:35:30.788248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:35:30.788248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:35:30.788248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:35:30.788248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:35:30.788248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:35:30.788248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:30.788248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 415/833] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:35:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:30.788927 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9d412-b0af-4c57-ae6c-3853bd0b8309 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:30.788927 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9d412-b0af-4c57-ae6c-3853bd0b8309 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:30.788927 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9d412-b0af-4c57-ae6c-3853bd0b8309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:30.806602 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-e9f9d412-b0af-4c57-ae6c-3853bd0b8309 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:30.807793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-e9f9d412-b0af-4c57-ae6c-3853bd0b8309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:30.814501 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f6331-495f-4754-a883-c0b201852a8f None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:30.852386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820ecaf-8b88-471a-a5ae-ab45c97753cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:30.853025 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820ecaf-8b88-471a-a5ae-ab45c97753cf None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:30.853328 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820ecaf-8b88-471a-a5ae-ab45c97753cf None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:30.853614 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820ecaf-8b88-471a-a5ae-ab45c97753cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:30.862430 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820ecaf-8b88-471a-a5ae-ab45c97753cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:30.863952 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820ecaf-8b88-471a-a5ae-ab45c97753cf None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:30.875933 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-0820ecaf-8b88-471a-a5ae-ab45c97753cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:30.875933 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:30.875933 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:30.875933 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:30.875933 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:30.875933 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:30.875933 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:30.875933 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:30.875933 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:30.875933 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:30.875933 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:30.875933 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:30.875933 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:35:30.875933 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:35:30.875933 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:35:30.876839 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:35:30.876839 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:30.876839 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:30.876839 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:30.876839 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:30.876839 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:30.876839 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:30.876839 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:30.876839 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:30.876839 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:30.876839 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:30.876839 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:30.876839 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:30.876839 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:30.876839 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:30.876839 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:30.876839 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:30.877776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:35:30.877776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:35:30.877776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:35:30.877776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:35:30.877776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:35:30.877776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:35:30.877776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:30.877776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:30.881634 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 419/834] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:35:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:30.882454 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820ecaf-8b88-471a-a5ae-ab45c97753cf None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:30.893804 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 416/835] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:30 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:35:30.893804 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f9d412-b0af-4c57-ae6c-3853bd0b8309 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:30.968758 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:30.998827 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820ecaf-8b88-471a-a5ae-ab45c97753cf None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:31.004640 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f9d412-b0af-4c57-ae6c-3853bd0b8309 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:31.023891 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:31.042482 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31945c0-7991-44f9-bb3a-fe78e291436e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:31.042782 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629f8f9-b76d-42e7-b670-72670cb2e614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:31.043099 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31945c0-7991-44f9-bb3a-fe78e291436e None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:31.043385 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31945c0-7991-44f9-bb3a-fe78e291436e None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:31.043660 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31945c0-7991-44f9-bb3a-fe78e291436e None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:31.043762 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629f8f9-b76d-42e7-b670-72670cb2e614 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:31.044112 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629f8f9-b76d-42e7-b670-72670cb2e614 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:31.044331 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31945c0-7991-44f9-bb3a-fe78e291436e None admin] RBAC: Adding query filter params (name=admin) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:31.044586 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629f8f9-b76d-42e7-b670-72670cb2e614 None admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:31.044683 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31945c0-7991-44f9-bb3a-fe78e291436e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:31.045472 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629f8f9-b76d-42e7-b670-72670cb2e614 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:31.046421 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629f8f9-b76d-42e7-b670-72670cb2e614 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:31.046698 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31945c0-7991-44f9-bb3a-fe78e291436e None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:31.048240 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629f8f9-b76d-42e7-b670-72670cb2e614 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:31.060711 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 420/836] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:35:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:31.060711 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31945c0-7991-44f9-bb3a-fe78e291436e None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:31.066978 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 417/837] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:35:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:31.072264 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629f8f9-b76d-42e7-b670-72670cb2e614 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:31.172135 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31945c0-7991-44f9-bb3a-fe78e291436e None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:31.179991 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629f8f9-b76d-42e7-b670-72670cb2e614 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:31.206400 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b054f96-f52b-4095-bf87-99734f10f160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:31.207035 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b054f96-f52b-4095-bf87-99734f10f160 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:31.207323 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b054f96-f52b-4095-bf87-99734f10f160 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:31.207603 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b054f96-f52b-4095-bf87-99734f10f160 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:31.208186 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b054f96-f52b-4095-bf87-99734f10f160 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:31.209589 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b054f96-f52b-4095-bf87-99734f10f160 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:31.213146 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:31.218402 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-7b054f96-f52b-4095-bf87-99734f10f160 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 22:35:31.218402 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:31.218402 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:35:31.218402 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:35:31.218402 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:35:31.218402 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:35:31.218402 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:35:31.218402 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:31.218402 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 22:35:31.218402 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:31.218402 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:35:31.218402 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:31.218402 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:31.218402 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:31.218402 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:31.218402 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:31.218402 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:31.219482 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:31.219482 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:31.219482 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:31.219482 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:31.219482 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:31.219482 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:31.219482 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:35:31.219482 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:35:31.219482 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:35:31.219482 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:35:31.219482 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:31.219482 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:31.219482 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:31.219482 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:31.219482 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:31.219482 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:31.220379 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:31.220379 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:31.220379 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:31.220379 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:31.220379 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:31.220379 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:31.220379 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:31.220379 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:31.220379 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:31.220379 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:31.220379 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:35:31.220379 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:31.220379 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 22:35:31.220379 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:31.222128 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c4d5b-5707-4886-a246-fd249b92b1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:31.222767 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c4d5b-5707-4886-a246-fd249b92b1dc None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:31.223047 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c4d5b-5707-4886-a246-fd249b92b1dc None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:31.223319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c4d5b-5707-4886-a246-fd249b92b1dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:31.224593 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 421/838] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 10 22:35:31 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:31.225469 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b054f96-f52b-4095-bf87-99734f10f160 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:31.236322 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6c4d5b-5707-4886-a246-fd249b92b1dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:31.238016 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6c4d5b-5707-4886-a246-fd249b92b1dc None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:31.255186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-de6c4d5b-5707-4886-a246-fd249b92b1dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:31.255186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:31.255186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:31.255186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:31.255186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:31.255186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:31.255186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:31.255186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:31.255186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:31.255186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:31.255186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:31.255186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:31.255186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:31.255186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:31.255186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:31.255186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:31.256168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:31.256168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:31.256168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:31.256168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:31.256168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:31.256168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:31.256168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:31.256168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:31.256168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:31.256168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:31.256168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:31.256168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:31.256168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:31.256168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:31.256168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:31.256168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:31.256168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:31.257070 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:31.257070 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:31.257070 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:31.257070 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 418/839] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:31.257328 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c4d5b-5707-4886-a246-fd249b92b1dc None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:31.288201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:31.345223 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b054f96-f52b-4095-bf87-99734f10f160 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:31.382131 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4483b-bec3-4afb-97c2-9fabbd5aa597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:31.382342 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c4d5b-5707-4886-a246-fd249b92b1dc None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:31.382895 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4483b-bec3-4afb-97c2-9fabbd5aa597 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:31.383246 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4483b-bec3-4afb-97c2-9fabbd5aa597 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:31.383588 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4483b-bec3-4afb-97c2-9fabbd5aa597 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:31.384381 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f4483b-bec3-4afb-97c2-9fabbd5aa597 None admin] RBAC: Adding query filter params (role.id=6641e95fdc1f4c098ebc372bb692d227, scope.project.id=3caa2ea846f7449b8c483788fc5846c8, user.id=b22838a8ad3949e59cdbaaa1801cd07f) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:31.384754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f4483b-bec3-4afb-97c2-9fabbd5aa597 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:31.386290 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f4483b-bec3-4afb-97c2-9fabbd5aa597 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:31.401758 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 422/840] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:35:31 2022] GET /identity/v3/role_assignments?user.id=b22838a8ad3949e59cdbaaa1801cd07f&scope.project.id=3caa2ea846f7449b8c483788fc5846c8&role.id=6641e95fdc1f4c098ebc372bb692d227 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:31.402567 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4483b-bec3-4afb-97c2-9fabbd5aa597 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:31.431827 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822e97d-fcd2-4158-b694-70e7ccb4ae9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:31.432456 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822e97d-fcd2-4158-b694-70e7ccb4ae9c None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:31.432729 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822e97d-fcd2-4158-b694-70e7ccb4ae9c None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:31.432994 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822e97d-fcd2-4158-b694-70e7ccb4ae9c None admin] PATH_INFO: `/v3/services` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:31.434096 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822e97d-fcd2-4158-b694-70e7ccb4ae9c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:31.434096 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822e97d-fcd2-4158-b694-70e7ccb4ae9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:31.435362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822e97d-fcd2-4158-b694-70e7ccb4ae9c None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:31.448198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 419/841] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 10 22:35:31 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:31.452118 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822e97d-fcd2-4158-b694-70e7ccb4ae9c None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:31.533230 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:31.555544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4483b-bec3-4afb-97c2-9fabbd5aa597 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:31.570434 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:31.595215 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b044b-9398-482a-8034-75ba7e14ef48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:31.596310 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b044b-9398-482a-8034-75ba7e14ef48 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:31.597026 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b044b-9398-482a-8034-75ba7e14ef48 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:31.597443 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b044b-9398-482a-8034-75ba7e14ef48 None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:31.598553 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b044b-9398-482a-8034-75ba7e14ef48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:31.598553 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b044b-9398-482a-8034-75ba7e14ef48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:31.600305 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b044b-9398-482a-8034-75ba7e14ef48 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:31.603877 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822e97d-fcd2-4158-b694-70e7ccb4ae9c None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:31.616967 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 423/842] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:31.623069 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b044b-9398-482a-8034-75ba7e14ef48 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:31.654805 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b38289-2ba8-4a8c-8a77-f291f41251fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:31.656727 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b38289-2ba8-4a8c-8a77-f291f41251fd None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:31.657008 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b38289-2ba8-4a8c-8a77-f291f41251fd None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:31.657352 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b38289-2ba8-4a8c-8a77-f291f41251fd None admin] PATH_INFO: `/v3/services` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:31.657946 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b38289-2ba8-4a8c-8a77-f291f41251fd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:31.658247 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b38289-2ba8-4a8c-8a77-f291f41251fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:31.659620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b38289-2ba8-4a8c-8a77-f291f41251fd None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:31.664459 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 420/843] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 10 22:35:31 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:31.750798 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b044b-9398-482a-8034-75ba7e14ef48 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:31.792025 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f778c555-2ec1-4fd1-a944-4bedd2d50a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:31.793068 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778c555-2ec1-4fd1-a944-4bedd2d50a92 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:31.793068 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778c555-2ec1-4fd1-a944-4bedd2d50a92 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:31.793366 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778c555-2ec1-4fd1-a944-4bedd2d50a92 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:31.802083 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f778c555-2ec1-4fd1-a944-4bedd2d50a92 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:31.803567 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f778c555-2ec1-4fd1-a944-4bedd2d50a92 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:31.816488 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-f778c555-2ec1-4fd1-a944-4bedd2d50a92 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 22:35:31.816488 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:31.816488 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:31.816488 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:31.816488 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:31.816488 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:31.816488 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:31.816488 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:31.816488 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:31.816488 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:31.816488 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:31.816488 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:31.816488 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:35:31.816488 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:35:31.816488 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:35:31.816488 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:35:31.817536 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:31.817536 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:31.817536 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:35:31.817536 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:31.817536 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:35:31.817536 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:31.817536 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:31.817536 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:31.817536 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:31.817536 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:31.817536 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:31.817536 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:31.817536 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:31.817536 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:31.817536 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:31.817536 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:31.817536 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:31.817536 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:31.819232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:31.819232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:31.819232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:35:31.819232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:35:31.819232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:35:31.819232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:35:31.819232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:35:31.819232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:35:31.819232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 22:35:31.819232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:31.819232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 424/844] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:35:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:31.824140 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b38289-2ba8-4a8c-8a77-f291f41251fd None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:31.938845 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b38289-2ba8-4a8c-8a77-f291f41251fd None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:31.953769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:31.976749 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60913290-cc92-42b5-a208-7c2fd1de5813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:31.977729 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60913290-cc92-42b5-a208-7c2fd1de5813 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:31.978220 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60913290-cc92-42b5-a208-7c2fd1de5813 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:31.978711 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60913290-cc92-42b5-a208-7c2fd1de5813 None admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:31.979805 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60913290-cc92-42b5-a208-7c2fd1de5813 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:31.980509 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60913290-cc92-42b5-a208-7c2fd1de5813 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:31.982447 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60913290-cc92-42b5-a208-7c2fd1de5813 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:32.004402 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 421/845] 10.222.0.76 () {56 vars in 1135 bytes} [Wed Aug 10 22:35:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:32.009703 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f778c555-2ec1-4fd1-a944-4bedd2d50a92 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:32.043364 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:32.116379 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f778c555-2ec1-4fd1-a944-4bedd2d50a92 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:32.151675 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb16ce4e-eb6f-4737-bfe0-fc64d47693e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:32.152296 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16ce4e-eb6f-4737-bfe0-fc64d47693e4 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:32.152574 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16ce4e-eb6f-4737-bfe0-fc64d47693e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:32.152849 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16ce4e-eb6f-4737-bfe0-fc64d47693e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:32.224605 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb16ce4e-eb6f-4737-bfe0-fc64d47693e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:32.226019 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb16ce4e-eb6f-4737-bfe0-fc64d47693e4 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:32.257703 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:32.330445 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 425/846] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:35:32 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 22:35:32.338379 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60913290-cc92-42b5-a208-7c2fd1de5813 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:32.446115 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60913290-cc92-42b5-a208-7c2fd1de5813 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:32.486713 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b5b99-0dab-4a97-affe-df339ed7a9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:32.487682 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b5b99-0dab-4a97-affe-df339ed7a9ce None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:32.488160 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b5b99-0dab-4a97-affe-df339ed7a9ce None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:32.488624 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b5b99-0dab-4a97-affe-df339ed7a9ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:32.500051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750b5b99-0dab-4a97-affe-df339ed7a9ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:32.501822 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750b5b99-0dab-4a97-affe-df339ed7a9ce None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:32.517394 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-750b5b99-0dab-4a97-affe-df339ed7a9ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:32.517394 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:32.517394 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:32.517394 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:32.517394 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:32.517394 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:32.517394 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:32.517394 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:32.517394 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:32.517394 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:32.517394 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:32.517394 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:32.517394 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:32.517394 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:32.517394 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:32.517394 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:32.518368 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:32.518368 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:32.518368 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:32.518368 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:32.518368 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:32.518368 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:32.518368 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:32.518368 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:32.518368 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:32.518368 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:32.518368 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:32.518368 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:32.518368 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:32.518368 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:32.518368 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:32.518368 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:32.518368 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:32.519254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:32.519254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:32.519254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:32.521414 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 422/847] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:32.532810 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb16ce4e-eb6f-4737-bfe0-fc64d47693e4 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:32.630575 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb16ce4e-eb6f-4737-bfe0-fc64d47693e4 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:32.661659 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:32.666915 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1380f7-e8b8-4dd1-9aeb-f67ae5ce9642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:32.667623 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1380f7-e8b8-4dd1-9aeb-f67ae5ce9642 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:32.668118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1380f7-e8b8-4dd1-9aeb-f67ae5ce9642 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:32.668791 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1380f7-e8b8-4dd1-9aeb-f67ae5ce9642 None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:32.669723 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1380f7-e8b8-4dd1-9aeb-f67ae5ce9642 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:32.670032 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1380f7-e8b8-4dd1-9aeb-f67ae5ce9642 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:32.671440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1380f7-e8b8-4dd1-9aeb-f67ae5ce9642 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:32.684157 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 426/848] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:35:32.689951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b5b99-0dab-4a97-affe-df339ed7a9ce None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:32.711505 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd44d9-837e-479c-bff1-3cd21622f5d5 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:32.711824 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd44d9-837e-479c-bff1-3cd21622f5d5 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:32.712048 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd44d9-837e-479c-bff1-3cd21622f5d5 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:32.713655 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 427/849] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:35:32.716992 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:32.719837 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb2705-349a-4246-9d33-0c3feb1bce14 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:32.720117 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb2705-349a-4246-9d33-0c3feb1bce14 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:32.720473 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb2705-349a-4246-9d33-0c3feb1bce14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:32.740883 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-13fb2705-349a-4246-9d33-0c3feb1bce14 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:32.742147 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-13fb2705-349a-4246-9d33-0c3feb1bce14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:32.804275 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b5b99-0dab-4a97-affe-df339ed7a9ce None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:32.820394 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 428/850] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:32 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:35:32.843879 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb2705-349a-4246-9d33-0c3feb1bce14 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:32.844086 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56426699-78e8-4d07-9d4f-42e0d408e5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:32.844086 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56426699-78e8-4d07-9d4f-42e0d408e5a9 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:32.844577 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56426699-78e8-4d07-9d4f-42e0d408e5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:32.844577 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56426699-78e8-4d07-9d4f-42e0d408e5a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:32.854377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56426699-78e8-4d07-9d4f-42e0d408e5a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:32.856434 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56426699-78e8-4d07-9d4f-42e0d408e5a9 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:32.870948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-56426699-78e8-4d07-9d4f-42e0d408e5a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:32.870948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:32.870948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:32.870948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:32.870948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:32.870948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:32.870948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:32.870948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:32.870948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:32.870948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:32.870948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:32.870948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:32.870948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:35:32.870948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:35:32.870948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:35:32.871846 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:35:32.871846 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:32.871846 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:32.871846 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:32.871846 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:32.871846 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:32.871846 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:32.871846 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:32.871846 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:32.871846 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:32.871846 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:32.871846 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:32.871846 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:32.871846 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:32.871846 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:32.871846 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:32.871846 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:32.872747 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:35:32.872747 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:35:32.872747 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:35:32.872747 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:35:32.872747 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:35:32.872747 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:35:32.872747 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:32.872747 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:32.873184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 423/851] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:35:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:32.878979 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56426699-78e8-4d07-9d4f-42e0d408e5a9 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:32.959812 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb2705-349a-4246-9d33-0c3feb1bce14 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:32.989594 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:33.001268 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56426699-78e8-4d07-9d4f-42e0d408e5a9 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:33.003423 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:33.006818 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ceb477-37b5-4c7d-822c-2efcf8b4e6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:33.007634 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ceb477-37b5-4c7d-822c-2efcf8b4e6d8 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:33.008130 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ceb477-37b5-4c7d-822c-2efcf8b4e6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:33.008556 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ceb477-37b5-4c7d-822c-2efcf8b4e6d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:33.023413 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ceb477-37b5-4c7d-822c-2efcf8b4e6d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:35:33.028955 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-50ceb477-37b5-4c7d-822c-2efcf8b4e6d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:35:33.028955 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:33.028955 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:33.028955 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:33.028955 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:33.028955 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:33.028955 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:33.028955 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:33.028955 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:33.028955 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:33.028955 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:33.028955 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:33.028955 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:35:33.028955 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:35:33.028955 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:35:33.028955 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:35:33.030071 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:35:33.030071 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:35:33.030071 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:35:33.030071 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:35:33.030071 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:33.030071 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:33.030071 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:33.030071 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:33.030071 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:33.030071 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:33.030071 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:33.030071 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:33.030071 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:33.030071 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:33.030071 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:33.030071 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:33.031128 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:33.031128 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:33.031128 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:33.031128 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:33.031128 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:35:33.031128 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:35:33.031128 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:35:33.031128 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:35:33.031128 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:35:33.031128 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:35:33.031128 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:35:33.031128 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:33.033951 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 429/852] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 10 22:35:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:35:33.040152 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ceb477-37b5-4c7d-822c-2efcf8b4e6d8 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:33.050665 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a102d2-1c49-43ea-bbe9-0ce9e4e715b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:33.051399 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a102d2-1c49-43ea-bbe9-0ce9e4e715b7 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:33.051692 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a102d2-1c49-43ea-bbe9-0ce9e4e715b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:33.051957 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a102d2-1c49-43ea-bbe9-0ce9e4e715b7 None admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:33.052654 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a102d2-1c49-43ea-bbe9-0ce9e4e715b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:33.052960 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a102d2-1c49-43ea-bbe9-0ce9e4e715b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:33.054540 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a102d2-1c49-43ea-bbe9-0ce9e4e715b7 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:33.069315 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 424/853] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:35:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:35:33.075837 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a102d2-1c49-43ea-bbe9-0ce9e4e715b7 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:33.176132 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ceb477-37b5-4c7d-822c-2efcf8b4e6d8 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:33.188160 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a102d2-1c49-43ea-bbe9-0ce9e4e715b7 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:33.225664 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663a12d-3303-49c8-90fa-68fbfb423d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:33.226523 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663a12d-3303-49c8-90fa-68fbfb423d6c None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:33.227018 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663a12d-3303-49c8-90fa-68fbfb423d6c None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:33.227400 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663a12d-3303-49c8-90fa-68fbfb423d6c None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:33.228350 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9663a12d-3303-49c8-90fa-68fbfb423d6c None admin] RBAC: Adding query filter params (name=service) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:33.228752 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9663a12d-3303-49c8-90fa-68fbfb423d6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:33.229168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c59654-9eab-4621-b78c-f4be3716270c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:33.230034 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c59654-9eab-4621-b78c-f4be3716270c None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:33.230464 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9663a12d-3303-49c8-90fa-68fbfb423d6c None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:33.230615 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c59654-9eab-4621-b78c-f4be3716270c None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:33.231028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c59654-9eab-4621-b78c-f4be3716270c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:33.231900 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c59654-9eab-4621-b78c-f4be3716270c None admin] RBAC: Adding query filter params (role.id=6641e95fdc1f4c098ebc372bb692d227, scope.project.id=3caa2ea846f7449b8c483788fc5846c8, user.id=1e3a76df95a34b9fb9816f22b85d1071) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:33.232406 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c59654-9eab-4621-b78c-f4be3716270c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:33.234854 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c59654-9eab-4621-b78c-f4be3716270c None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:33.246385 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 430/855] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 10 22:35:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:33.246624 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 425/855] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:35:33 2022] GET /identity/v3/role_assignments?user.id=1e3a76df95a34b9fb9816f22b85d1071&scope.project.id=3caa2ea846f7449b8c483788fc5846c8&role.id=6641e95fdc1f4c098ebc372bb692d227 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:35:33.247627 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663a12d-3303-49c8-90fa-68fbfb423d6c None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:33.303636 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:33.364514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663a12d-3303-49c8-90fa-68fbfb423d6c None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:33.402818 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c71b2b9-92c8-4fc4-8b00-5279468c7d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:33.405407 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71b2b9-92c8-4fc4-8b00-5279468c7d0a None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:33.405407 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71b2b9-92c8-4fc4-8b00-5279468c7d0a None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:33.405407 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71b2b9-92c8-4fc4-8b00-5279468c7d0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:33.415421 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c71b2b9-92c8-4fc4-8b00-5279468c7d0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:33.416881 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c71b2b9-92c8-4fc4-8b00-5279468c7d0a None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:33.432770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-1c71b2b9-92c8-4fc4-8b00-5279468c7d0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:33.432770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:33.432770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:33.432770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:33.432770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:33.432770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:33.432770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:33.432770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:33.432770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:33.432770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:33.432770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:33.432770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:33.432770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:33.432770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:33.432770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:33.432770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:33.434003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:33.434003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:33.434003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:33.434003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:33.434003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:33.434003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:33.434003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:33.434003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:33.434003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:33.434003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:33.434003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:33.434003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:33.434003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:33.434003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:33.434003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:33.434003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:33.434003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:33.434898 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:33.434898 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:33.434898 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:33.434898 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 431/856] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:33.440031 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c59654-9eab-4621-b78c-f4be3716270c None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:33.454494 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:33.550148 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c59654-9eab-4621-b78c-f4be3716270c None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:33.586158 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0106bebc-a3fd-480d-92a9-ad863163bc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:33.587027 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106bebc-a3fd-480d-92a9-ad863163bc42 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:33.587455 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106bebc-a3fd-480d-92a9-ad863163bc42 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:33.587874 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106bebc-a3fd-480d-92a9-ad863163bc42 None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:33.588660 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0106bebc-a3fd-480d-92a9-ad863163bc42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:33.589111 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0106bebc-a3fd-480d-92a9-ad863163bc42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:33.590666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0106bebc-a3fd-480d-92a9-ad863163bc42 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:33.604175 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 426/857] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:33.609921 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c71b2b9-92c8-4fc4-8b00-5279468c7d0a None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:33.627350 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61430473-8db2-4fb3-b355-87ab9cae202e None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:33.627866 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61430473-8db2-4fb3-b355-87ab9cae202e None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:33.628356 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61430473-8db2-4fb3-b355-87ab9cae202e None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:33.630268 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 427/858] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:35:33.635957 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c274f2-6938-4c0d-bce4-3f04dec16e29 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:33.636349 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c274f2-6938-4c0d-bce4-3f04dec16e29 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:33.636718 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c274f2-6938-4c0d-bce4-3f04dec16e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:33.656506 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-22c274f2-6938-4c0d-bce4-3f04dec16e29 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:33.657735 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-22c274f2-6938-4c0d-bce4-3f04dec16e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:33.719202 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:33.725648 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:33.726021 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c71b2b9-92c8-4fc4-8b00-5279468c7d0a None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:33.745649 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 428/859] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:33 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:35:33.759971 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c274f2-6938-4c0d-bce4-3f04dec16e29 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:33.762843 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c60f8-7bc6-41c9-a6ad-1c0dd7021667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:33.763644 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c60f8-7bc6-41c9-a6ad-1c0dd7021667 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:33.764188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c60f8-7bc6-41c9-a6ad-1c0dd7021667 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:33.764519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c60f8-7bc6-41c9-a6ad-1c0dd7021667 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:33.772786 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c60f8-7bc6-41c9-a6ad-1c0dd7021667 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:33.774341 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c60f8-7bc6-41c9-a6ad-1c0dd7021667 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:33.786954 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-846c60f8-7bc6-41c9-a6ad-1c0dd7021667 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 22:35:33.786954 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:33.786954 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:33.786954 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:33.786954 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:33.786954 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:33.786954 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:33.786954 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:33.786954 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:33.786954 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:33.786954 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:33.786954 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:33.786954 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:35:33.786954 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:35:33.786954 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:35:33.786954 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:35:33.787934 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:33.787934 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:33.787934 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:35:33.787934 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:33.787934 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:35:33.787934 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:33.787934 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:33.787934 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:33.787934 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:33.787934 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:33.787934 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:33.787934 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:33.787934 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:33.787934 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:33.787934 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:33.787934 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:33.787934 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:33.787934 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:33.788871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:33.788871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:33.788871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:35:33.788871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:35:33.788871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:35:33.788871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:35:33.788871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:35:33.788871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:35:33.788871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 22:35:33.788871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:33.788871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 432/860] 10.222.0.76 () {56 vars in 1117 bytes} [Wed Aug 10 22:35:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:33.793990 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c60f8-7bc6-41c9-a6ad-1c0dd7021667 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:33.878373 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c274f2-6938-4c0d-bce4-3f04dec16e29 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:33.903376 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c60f8-7bc6-41c9-a6ad-1c0dd7021667 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:33.914785 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24caefa5-a4d1-4523-9e2d-01142ccef971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:33.915417 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24caefa5-a4d1-4523-9e2d-01142ccef971 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:33.915696 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24caefa5-a4d1-4523-9e2d-01142ccef971 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:33.916072 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24caefa5-a4d1-4523-9e2d-01142ccef971 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:33.916662 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24caefa5-a4d1-4523-9e2d-01142ccef971 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:33.918073 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24caefa5-a4d1-4523-9e2d-01142ccef971 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:33.928861 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-24caefa5-a4d1-4523-9e2d-01142ccef971 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 22:35:33.928861 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:33.928861 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:35:33.928861 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:35:33.928861 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:35:33.928861 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:35:33.928861 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:35:33.928861 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:33.928861 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 22:35:33.928861 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:33.928861 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:35:33.928861 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:33.928861 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:33.928861 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:33.928861 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:33.928861 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:33.930062 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:33.930062 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:33.930062 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:33.930062 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:33.930062 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:33.930062 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:33.930062 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:33.930062 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:35:33.930062 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:35:33.930062 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:35:33.930062 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:35:33.930062 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:33.930062 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:33.930062 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:33.930062 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:33.930062 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:33.930062 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:33.931249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:33.931249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:33.931249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:33.931249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:33.931249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:33.931249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:33.931249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:33.931249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:33.931249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:33.931249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:33.931249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:35:33.931249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:33.931249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 22:35:33.931249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:33.932019 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 429/861] 10.222.0.76 () {56 vars in 1132 bytes} [Wed Aug 10 22:35:33 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:35:33.932889 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8533d-eb77-4a48-b19e-faea505cb5e3 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:33.933471 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8533d-eb77-4a48-b19e-faea505cb5e3 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:33.933471 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8533d-eb77-4a48-b19e-faea505cb5e3 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:33.934784 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 430/862] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:33.938267 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8533d-eb77-4a48-b19e-faea505cb5e3 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:33.951309 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830467d-422a-480b-b79b-3ff9bdf56c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:33.952784 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830467d-422a-480b-b79b-3ff9bdf56c13 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:33.953084 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830467d-422a-480b-b79b-3ff9bdf56c13 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:33.953395 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830467d-422a-480b-b79b-3ff9bdf56c13 None admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:33.954134 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b830467d-422a-480b-b79b-3ff9bdf56c13 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:33.954442 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b830467d-422a-480b-b79b-3ff9bdf56c13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:33.955883 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b830467d-422a-480b-b79b-3ff9bdf56c13 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:33.978300 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 433/863] 10.222.0.76 () {56 vars in 1149 bytes} [Wed Aug 10 22:35:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:35:33.978300 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddcbae6-f4fc-41e5-9814-73fdbdaf78e6 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:33.978300 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddcbae6-f4fc-41e5-9814-73fdbdaf78e6 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:33.978300 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddcbae6-f4fc-41e5-9814-73fdbdaf78e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:33.995372 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-5ddcbae6-f4fc-41e5-9814-73fdbdaf78e6 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:33.996675 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-5ddcbae6-f4fc-41e5-9814-73fdbdaf78e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:34.008375 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:34.011819 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:34.055072 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8533d-eb77-4a48-b19e-faea505cb5e3 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:34.081501 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 434/864] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:33 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:35:34.081501 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddcbae6-f4fc-41e5-9814-73fdbdaf78e6 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:34.094339 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2753a15-c045-4024-8a72-d9aaa2387d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:34.095093 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2753a15-c045-4024-8a72-d9aaa2387d85 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:34.095436 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2753a15-c045-4024-8a72-d9aaa2387d85 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:34.095777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2753a15-c045-4024-8a72-d9aaa2387d85 None admin] PATH_INFO: `/v3/services` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:34.096541 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2753a15-c045-4024-8a72-d9aaa2387d85 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:34.096906 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2753a15-c045-4024-8a72-d9aaa2387d85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:34.098573 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2753a15-c045-4024-8a72-d9aaa2387d85 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:34.103760 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 431/865] 10.222.0.76 () {56 vars in 1126 bytes} [Wed Aug 10 22:35:33 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:34.104608 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2753a15-c045-4024-8a72-d9aaa2387d85 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:34.205656 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddcbae6-f4fc-41e5-9814-73fdbdaf78e6 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:34.211379 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2753a15-c045-4024-8a72-d9aaa2387d85 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:34.236347 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:34.243088 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b68b87d-a0b7-4416-94e2-361449d90131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:34.243697 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68b87d-a0b7-4416-94e2-361449d90131 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:34.243979 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68b87d-a0b7-4416-94e2-361449d90131 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:34.244248 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68b87d-a0b7-4416-94e2-361449d90131 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:34.248598 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a3be1-6c98-4d30-8b2a-c73c09990541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:34.249294 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a3be1-6c98-4d30-8b2a-c73c09990541 None admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:34.249572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a3be1-6c98-4d30-8b2a-c73c09990541 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:34.249840 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a3be1-6c98-4d30-8b2a-c73c09990541 None admin] PATH_INFO: `/v3/services` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:34.250473 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3a3be1-6c98-4d30-8b2a-c73c09990541 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:34.251829 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3a3be1-6c98-4d30-8b2a-c73c09990541 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:34.260175 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:34.267574 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 432/866] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 10 22:35:34 2022] POST /identity/v3/services => generated 251 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:35:34.274312 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a3be1-6c98-4d30-8b2a-c73c09990541 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:34.324818 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b68b87d-a0b7-4416-94e2-361449d90131 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:34.326533 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b68b87d-a0b7-4416-94e2-361449d90131 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:34.410051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a3be1-6c98-4d30-8b2a-c73c09990541 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:34.442753 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 435/867] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:35:34 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 22:35:34.450514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b68b87d-a0b7-4416-94e2-361449d90131 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:34.469508 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd70a3-8fd2-4ae2-82c8-05f8e12712ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:34.469508 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd70a3-8fd2-4ae2-82c8-05f8e12712ec None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:34.469776 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd70a3-8fd2-4ae2-82c8-05f8e12712ec None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:34.470002 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd70a3-8fd2-4ae2-82c8-05f8e12712ec None admin] PATH_INFO: `/v3/services` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:34.470612 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddd70a3-8fd2-4ae2-82c8-05f8e12712ec None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:34.470913 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddd70a3-8fd2-4ae2-82c8-05f8e12712ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:34.472307 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddd70a3-8fd2-4ae2-82c8-05f8e12712ec None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:34.478178 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 433/868] 10.222.0.76 () {56 vars in 1126 bytes} [Wed Aug 10 22:35:34 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:34.516645 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:34.583576 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b68b87d-a0b7-4416-94e2-361449d90131 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:34.621589 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7bb2b0-bf3b-4fd6-961c-ac007ff16c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:34.622245 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bb2b0-bf3b-4fd6-961c-ac007ff16c6b None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:34.622527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bb2b0-bf3b-4fd6-961c-ac007ff16c6b None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:34.622801 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bb2b0-bf3b-4fd6-961c-ac007ff16c6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:34.635284 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7bb2b0-bf3b-4fd6-961c-ac007ff16c6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:34.636792 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7bb2b0-bf3b-4fd6-961c-ac007ff16c6b None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:34.652516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-6a7bb2b0-bf3b-4fd6-961c-ac007ff16c6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:34.652516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:34.652516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:34.652516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:34.652516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:34.652516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:34.652516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:34.652516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:34.652516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:34.652516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:34.652516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:34.652516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:34.652516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:34.652516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:34.652516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:34.652516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:34.653516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:34.653516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:34.653516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:34.653516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:34.653516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:34.653516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:34.653516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:34.653516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:34.653516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:34.653516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:34.653516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:34.653516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:34.653516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:34.653516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:34.653516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:34.653516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:34.653516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:34.654679 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:34.654679 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:34.654679 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:34.654679 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 436/869] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:34.659654 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd70a3-8fd2-4ae2-82c8-05f8e12712ec None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:34.713733 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:34.769804 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd70a3-8fd2-4ae2-82c8-05f8e12712ec None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:34.804106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623111f3-c80b-467f-8861-52fe64a13fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:34.804717 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623111f3-c80b-467f-8861-52fe64a13fd5 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:34.804996 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623111f3-c80b-467f-8861-52fe64a13fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:34.805267 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623111f3-c80b-467f-8861-52fe64a13fd5 None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:34.805988 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623111f3-c80b-467f-8861-52fe64a13fd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:34.806291 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623111f3-c80b-467f-8861-52fe64a13fd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:34.807717 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623111f3-c80b-467f-8861-52fe64a13fd5 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:34.821261 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 434/870] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:34.826800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7bb2b0-bf3b-4fd6-961c-ac007ff16c6b None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:34.942083 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:34.948772 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7bb2b0-bf3b-4fd6-961c-ac007ff16c6b None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:34.986592 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bcded0-5d2b-4342-9ef0-dec1f49a19d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:34.987633 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bcded0-5d2b-4342-9ef0-dec1f49a19d8 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:34.988139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bcded0-5d2b-4342-9ef0-dec1f49a19d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:34.988607 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bcded0-5d2b-4342-9ef0-dec1f49a19d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:34.995972 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bcded0-5d2b-4342-9ef0-dec1f49a19d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:34.998459 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bcded0-5d2b-4342-9ef0-dec1f49a19d8 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:35.010655 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-37bcded0-5d2b-4342-9ef0-dec1f49a19d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:35.010655 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:35.010655 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:35.010655 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:35.010655 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:35.010655 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:35.010655 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:35.010655 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:35.010655 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:35.010655 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:35.010655 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:35.010655 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:35.010655 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:35:35.010655 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:35:35.010655 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:35:35.011668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:35:35.011668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:35.011668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:35.011668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:35.011668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:35.011668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:35.011668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:35.011668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:35.011668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:35.011668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:35.011668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:35.011668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:35.011668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:35.011668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:35.011668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:35.011668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:35.011668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:35.012538 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:35:35.012538 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:35:35.012538 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:35:35.012538 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:35:35.012538 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:35:35.012538 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:35:35.012538 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:35.012538 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:35.015545 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 437/871] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:35:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:35.022251 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623111f3-c80b-467f-8861-52fe64a13fd5 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:35.138123 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623111f3-c80b-467f-8861-52fe64a13fd5 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:35.157654 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:35.178377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc98860e-8700-4b9d-9dd2-41c6e37b9b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:35.179327 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98860e-8700-4b9d-9dd2-41c6e37b9b58 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:35.179804 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98860e-8700-4b9d-9dd2-41c6e37b9b58 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:35.180300 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98860e-8700-4b9d-9dd2-41c6e37b9b58 None admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:35.181240 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc98860e-8700-4b9d-9dd2-41c6e37b9b58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:35.181835 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc98860e-8700-4b9d-9dd2-41c6e37b9b58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:35.183477 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc98860e-8700-4b9d-9dd2-41c6e37b9b58 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:35.197705 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 435/872] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:35:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:35.205329 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bcded0-5d2b-4342-9ef0-dec1f49a19d8 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:35.306993 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bcded0-5d2b-4342-9ef0-dec1f49a19d8 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:35.343054 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10ef7e-5662-4ab8-9efe-8dff48d4e824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:35.343661 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10ef7e-5662-4ab8-9efe-8dff48d4e824 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:35.343935 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10ef7e-5662-4ab8-9efe-8dff48d4e824 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:35.344224 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10ef7e-5662-4ab8-9efe-8dff48d4e824 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:35.344928 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f10ef7e-5662-4ab8-9efe-8dff48d4e824 None admin] RBAC: Adding query filter params (role.id=5dfd0f952cd641bbb2257860c4c892ac, scope.project.id=3caa2ea846f7449b8c483788fc5846c8, user.id=aff5e804c64345b298c745287a1d0a32) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:35.345230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f10ef7e-5662-4ab8-9efe-8dff48d4e824 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:35.346682 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f10ef7e-5662-4ab8-9efe-8dff48d4e824 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:35.352761 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 438/873] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:35:35 2022] GET /identity/v3/role_assignments?user.id=aff5e804c64345b298c745287a1d0a32&scope.project.id=3caa2ea846f7449b8c483788fc5846c8&role.id=5dfd0f952cd641bbb2257860c4c892ac => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:35.477880 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6ec23-1434-4a67-a23e-189f5efc633d None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:35.478219 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6ec23-1434-4a67-a23e-189f5efc633d None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:35.478493 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6ec23-1434-4a67-a23e-189f5efc633d None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:35.480198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 436/874] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:35.486493 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae6ce3-1aba-46b4-a1bf-7f8371233e47 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:35.486791 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae6ce3-1aba-46b4-a1bf-7f8371233e47 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:35.487044 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae6ce3-1aba-46b4-a1bf-7f8371233e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:35.508039 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-d5ae6ce3-1aba-46b4-a1bf-7f8371233e47 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:35.509258 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-d5ae6ce3-1aba-46b4-a1bf-7f8371233e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:35.525499 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:35.588988 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 439/875] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:35 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:35:35.611649 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6ec23-1434-4a67-a23e-189f5efc633d None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:35.732983 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6ec23-1434-4a67-a23e-189f5efc633d None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:35.769289 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3e277-c928-4385-b2e4-dd932169907d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:35.769981 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3e277-c928-4385-b2e4-dd932169907d None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:35.770258 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3e277-c928-4385-b2e4-dd932169907d None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:35.770526 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3e277-c928-4385-b2e4-dd932169907d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:35.771103 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d3e277-c928-4385-b2e4-dd932169907d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:35.772529 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d3e277-c928-4385-b2e4-dd932169907d None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:35.782260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-a8d3e277-c928-4385-b2e4-dd932169907d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 22:35:35.782260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:35.782260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:35:35.782260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:35:35.782260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:35:35.782260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:35:35.782260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:35:35.782260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:35.782260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 22:35:35.782260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:35.782260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:35:35.782260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:35.782260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:35.782260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:35.782260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:35.782260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:35.782260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:35.783259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:35.783259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:35.783259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:35.783259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:35.783259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:35.783259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:35.783259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:35:35.783259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:35:35.783259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:35:35.783259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:35:35.783259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:35.783259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:35.783259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:35.783259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:35.783259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:35.783259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:35.784724 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:35.784724 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:35.784724 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:35.784724 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:35.784724 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:35.784724 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:35.784724 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:35.784724 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:35.784724 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:35.784724 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:35.784724 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:35:35.784724 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:35.784724 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 22:35:35.784724 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:35.784724 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 437/876] 10.222.0.76 () {56 vars in 1126 bytes} [Wed Aug 10 22:35:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:35:35.789264 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ae6ce3-1aba-46b4-a1bf-7f8371233e47 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:35.897803 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ae6ce3-1aba-46b4-a1bf-7f8371233e47 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:35.933872 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe761bc-5607-4970-a46b-54341ae25e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:35.934785 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe761bc-5607-4970-a46b-54341ae25e12 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:35.935778 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe761bc-5607-4970-a46b-54341ae25e12 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:35.936246 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe761bc-5607-4970-a46b-54341ae25e12 None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:35.937056 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe761bc-5607-4970-a46b-54341ae25e12 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:35.937556 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe761bc-5607-4970-a46b-54341ae25e12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:35.939161 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe761bc-5607-4970-a46b-54341ae25e12 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:35.944953 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 440/877] 10.222.0.76 () {56 vars in 1122 bytes} [Wed Aug 10 22:35:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:35.951724 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3e277-c928-4385-b2e4-dd932169907d None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:35.983968 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:36.060961 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3e277-c928-4385-b2e4-dd932169907d None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:36.097129 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709d0f64-a867-4230-a885-8118c66109f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:36.098106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d0f64-a867-4230-a885-8118c66109f4 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:36.098572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d0f64-a867-4230-a885-8118c66109f4 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:36.099044 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d0f64-a867-4230-a885-8118c66109f4 None admin] PATH_INFO: `/v3/services` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:36.099869 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709d0f64-a867-4230-a885-8118c66109f4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:36.100439 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709d0f64-a867-4230-a885-8118c66109f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:36.102131 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709d0f64-a867-4230-a885-8118c66109f4 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:36.108664 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 438/878] 10.222.0.76 () {56 vars in 1122 bytes} [Wed Aug 10 22:35:35 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:36.543855 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba70e3f-912c-4901-a18b-96ecbe1443b6 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:36.544270 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba70e3f-912c-4901-a18b-96ecbe1443b6 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:36.544568 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba70e3f-912c-4901-a18b-96ecbe1443b6 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:36.546559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 441/879] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:36.552077 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbea699-1759-4e2e-993d-bfa3db5ef141 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:36.552366 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbea699-1759-4e2e-993d-bfa3db5ef141 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:36.552640 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbea699-1759-4e2e-993d-bfa3db5ef141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:36.572240 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-dfbea699-1759-4e2e-993d-bfa3db5ef141 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:36.573804 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-dfbea699-1759-4e2e-993d-bfa3db5ef141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:36.644507 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d30c8-fd16-4bd2-9473-c7c660378e69 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:36.644733 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d30c8-fd16-4bd2-9473-c7c660378e69 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:36.645014 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d30c8-fd16-4bd2-9473-c7c660378e69 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:36.646641 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 442/880] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:36.651599 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0248d-ecbd-4d6c-bb96-24cd20618bbb None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:36.651849 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0248d-ecbd-4d6c-bb96-24cd20618bbb None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:36.652189 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0248d-ecbd-4d6c-bb96-24cd20618bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:36.652393 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 439/881] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:36 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:35:36.672992 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-d1c0248d-ecbd-4d6c-bb96-24cd20618bbb None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:36.673957 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbea699-1759-4e2e-993d-bfa3db5ef141 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:36.674224 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-d1c0248d-ecbd-4d6c-bb96-24cd20618bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:36.685622 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:36.708981 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:36.755840 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 443/882] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:36 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:35:36.778682 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0248d-ecbd-4d6c-bb96-24cd20618bbb None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:36.806574 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbea699-1759-4e2e-993d-bfa3db5ef141 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:36.848896 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77402006-849c-488c-9713-b3c6fa7e002e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:36.849959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77402006-849c-488c-9713-b3c6fa7e002e None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:36.850446 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77402006-849c-488c-9713-b3c6fa7e002e None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:36.851032 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77402006-849c-488c-9713-b3c6fa7e002e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:36.851914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77402006-849c-488c-9713-b3c6fa7e002e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:36.853498 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77402006-849c-488c-9713-b3c6fa7e002e None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:36.863250 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-77402006-849c-488c-9713-b3c6fa7e002e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 22:35:36.863250 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:36.863250 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:35:36.863250 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:35:36.863250 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:35:36.863250 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:35:36.863250 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:35:36.863250 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:36.863250 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 22:35:36.863250 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:36.863250 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:35:36.863250 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:36.863250 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:36.863250 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:36.863250 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:36.863250 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:36.863250 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:36.864233 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:36.864233 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:36.864233 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:36.864233 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:36.864233 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:36.864233 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:36.864233 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:35:36.864233 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:35:36.864233 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:35:36.864233 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:35:36.864233 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:36.864233 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:36.864233 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:36.864233 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:36.864233 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:36.864233 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:36.865064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:36.865064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:36.865064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:36.865064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:36.865064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:36.865064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:36.865064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:36.865064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:36.865064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:36.865064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:36.865064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:35:36.865064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:36.865064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 22:35:36.865064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:36.867980 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 440/883] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 10 22:35:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:35:36.875075 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77402006-849c-488c-9713-b3c6fa7e002e None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:36.892750 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0248d-ecbd-4d6c-bb96-24cd20618bbb None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:36.931799 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134d100-6b88-4ef3-8c57-d71af5193ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:36.932419 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134d100-6b88-4ef3-8c57-d71af5193ddc None admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:36.932698 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134d100-6b88-4ef3-8c57-d71af5193ddc None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:36.932970 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134d100-6b88-4ef3-8c57-d71af5193ddc None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:36.933713 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9134d100-6b88-4ef3-8c57-d71af5193ddc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:36.935710 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9134d100-6b88-4ef3-8c57-d71af5193ddc None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:36.958076 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 444/884] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 10 22:35:36 2022] POST /identity/v3/services => generated 269 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:35:36.983349 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:37.002717 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77402006-849c-488c-9713-b3c6fa7e002e None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:37.052145 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e0cbe-08fa-4662-ab9e-b6156b957dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:37.053057 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e0cbe-08fa-4662-ab9e-b6156b957dc9 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:37.053501 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e0cbe-08fa-4662-ab9e-b6156b957dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:37.053968 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e0cbe-08fa-4662-ab9e-b6156b957dc9 None admin] PATH_INFO: `/v3/services` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:37.054811 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e0cbe-08fa-4662-ab9e-b6156b957dc9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:37.055284 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e0cbe-08fa-4662-ab9e-b6156b957dc9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:37.056880 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e0cbe-08fa-4662-ab9e-b6156b957dc9 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:37.067822 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 441/885] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 10 22:35:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:35:37.070212 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134d100-6b88-4ef3-8c57-d71af5193ddc None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:37.074595 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:37.190742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134d100-6b88-4ef3-8c57-d71af5193ddc None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:37.228136 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714356f-06c9-45c9-a22a-51b77263fa9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:37.228908 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714356f-06c9-45c9-a22a-51b77263fa9a None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:37.229190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714356f-06c9-45c9-a22a-51b77263fa9a None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:37.229519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714356f-06c9-45c9-a22a-51b77263fa9a None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:37.230204 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714356f-06c9-45c9-a22a-51b77263fa9a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:37.230513 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714356f-06c9-45c9-a22a-51b77263fa9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:37.232700 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714356f-06c9-45c9-a22a-51b77263fa9a None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:37.238171 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 445/886] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 10 22:35:37 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:35:37.244293 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e0cbe-08fa-4662-ab9e-b6156b957dc9 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:37.356466 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e0cbe-08fa-4662-ab9e-b6156b957dc9 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:37.393483 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96fa37-f2db-4aa0-8e3e-0dc171a2d31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:37.394488 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96fa37-f2db-4aa0-8e3e-0dc171a2d31d None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:37.394981 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96fa37-f2db-4aa0-8e3e-0dc171a2d31d None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:37.395451 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96fa37-f2db-4aa0-8e3e-0dc171a2d31d None admin] PATH_INFO: `/v3/endpoints` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:37.396357 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd96fa37-f2db-4aa0-8e3e-0dc171a2d31d None admin] RBAC: Adding query filter params (interface=public, service_id=2d6d317c523a49c9b2756e43fcf7de68, region_id=RegionOne) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:37.396866 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd96fa37-f2db-4aa0-8e3e-0dc171a2d31d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:37.398448 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd96fa37-f2db-4aa0-8e3e-0dc171a2d31d None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:37.410701 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 442/887] 10.222.0.76 () {56 vars in 1251 bytes} [Wed Aug 10 22:35:37 2022] GET /identity/v3/endpoints?service_id=2d6d317c523a49c9b2756e43fcf7de68&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:37.412314 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714356f-06c9-45c9-a22a-51b77263fa9a None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:37.486382 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:37.523831 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714356f-06c9-45c9-a22a-51b77263fa9a None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:37.558974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb4de3-4cab-4b40-9298-09006a5b96d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:37.560130 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb4de3-4cab-4b40-9298-09006a5b96d9 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:37.560409 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb4de3-4cab-4b40-9298-09006a5b96d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:37.560763 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb4de3-4cab-4b40-9298-09006a5b96d9 None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:37.561506 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debb4de3-4cab-4b40-9298-09006a5b96d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:37.562944 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debb4de3-4cab-4b40-9298-09006a5b96d9 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:37.573391 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 446/888] 10.222.0.76 () {56 vars in 1087 bytes} [Wed Aug 10 22:35:37 2022] GET /identity/v3/services => generated 1268 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:35:37.579150 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51e8d1-8cd1-47c3-a519-774926f82658 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:37.579486 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51e8d1-8cd1-47c3-a519-774926f82658 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:37.579712 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51e8d1-8cd1-47c3-a519-774926f82658 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:37.581455 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 443/889] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:37.587359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34032dd4-4f1b-46cb-a074-6553e017a27d None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:37.587672 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34032dd4-4f1b-46cb-a074-6553e017a27d None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:37.587928 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34032dd4-4f1b-46cb-a074-6553e017a27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:37.609610 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-34032dd4-4f1b-46cb-a074-6553e017a27d None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:37.610840 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-34032dd4-4f1b-46cb-a074-6553e017a27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:37.689732 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 447/890] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:37 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:35:37.714672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51e8d1-8cd1-47c3-a519-774926f82658 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:37.730278 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:37.835070 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51e8d1-8cd1-47c3-a519-774926f82658 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:37.874446 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39118c-da47-4566-836e-566460c9f835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:37.875134 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39118c-da47-4566-836e-566460c9f835 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:37.875415 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39118c-da47-4566-836e-566460c9f835 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:37.875684 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39118c-da47-4566-836e-566460c9f835 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:37.888059 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39118c-da47-4566-836e-566460c9f835 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:35:37.892959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-6c39118c-da47-4566-836e-566460c9f835 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 22:35:37.892959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:37.892959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:37.892959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:37.892959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:37.892959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:37.892959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:37.892959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:37.892959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:37.892959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:37.892959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:37.892959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:37.892959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:35:37.892959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:35:37.892959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:35:37.892959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:35:37.893934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:35:37.893934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:35:37.893934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:35:37.893934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:35:37.893934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:37.893934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:37.893934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:37.893934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:37.893934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:37.893934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:37.893934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:37.893934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:37.893934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:37.893934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:37.893934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:37.893934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:37.894775 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:37.894775 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:37.894775 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:37.894775 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:37.894775 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:35:37.894775 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:35:37.894775 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:35:37.894775 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:35:37.894775 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:35:37.894775 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:35:37.894775 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 22:35:37.894775 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:37.894775 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 444/891] 10.222.0.76 () {56 vars in 1120 bytes} [Wed Aug 10 22:35:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:37.900187 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34032dd4-4f1b-46cb-a074-6553e017a27d None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:37.967014 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:38.008840 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34032dd4-4f1b-46cb-a074-6553e017a27d None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:38.045247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d714378-21a0-4808-af50-15b710e0f2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:38.045887 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d714378-21a0-4808-af50-15b710e0f2f1 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:38.046160 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d714378-21a0-4808-af50-15b710e0f2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:38.046428 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d714378-21a0-4808-af50-15b710e0f2f1 None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:38.047082 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d714378-21a0-4808-af50-15b710e0f2f1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:38.047377 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d714378-21a0-4808-af50-15b710e0f2f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:38.048740 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d714378-21a0-4808-af50-15b710e0f2f1 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:38.057201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 448/892] 10.222.0.76 () {56 vars in 1115 bytes} [Wed Aug 10 22:35:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:38.062954 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39118c-da47-4566-836e-566460c9f835 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:38.140282 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:38.174603 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39118c-da47-4566-836e-566460c9f835 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:38.213916 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49371585-c2d9-4f95-968c-c492c0d54535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:38.214552 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49371585-c2d9-4f95-968c-c492c0d54535 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:38.214831 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49371585-c2d9-4f95-968c-c492c0d54535 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:38.215102 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49371585-c2d9-4f95-968c-c492c0d54535 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:38.226285 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49371585-c2d9-4f95-968c-c492c0d54535 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:38.227745 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49371585-c2d9-4f95-968c-c492c0d54535 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:38.244179 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-49371585-c2d9-4f95-968c-c492c0d54535 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:38.244179 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:38.244179 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:38.244179 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:38.244179 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:38.244179 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:38.244179 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:38.244179 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:38.244179 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:38.244179 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:38.244179 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:38.244179 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:38.244179 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:38.244179 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:38.244179 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:38.244179 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:38.245168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:38.245168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:38.245168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:38.245168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:38.245168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:38.245168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:38.245168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:38.245168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:38.245168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:38.245168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:38.245168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:38.245168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:38.245168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:38.245168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:38.245168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:38.245168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:38.245168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:38.246106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:38.246106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:38.246106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:38.246106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 445/893] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:38.254008 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d714378-21a0-4808-af50-15b710e0f2f1 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:38.258892 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6770881-28f6-45c9-8a7e-d445ef278679 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:38.259224 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6770881-28f6-45c9-8a7e-d445ef278679 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:38.259450 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6770881-28f6-45c9-8a7e-d445ef278679 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:38.260932 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 446/894] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:35:38.271041 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82015eb-0b04-416a-b913-5c35727af74d None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:38.271387 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82015eb-0b04-416a-b913-5c35727af74d None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:38.271647 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82015eb-0b04-416a-b913-5c35727af74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:38.292479 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-f82015eb-0b04-416a-b913-5c35727af74d None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:38.293989 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-f82015eb-0b04-416a-b913-5c35727af74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:38.372795 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d714378-21a0-4808-af50-15b710e0f2f1 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:38.379464 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 447/895] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:38 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:35:38.397395 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:38.398914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82015eb-0b04-416a-b913-5c35727af74d None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:38.409260 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7eaaa-58f4-4566-9789-0387a50da771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:38.409918 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7eaaa-58f4-4566-9789-0387a50da771 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:38.410420 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7eaaa-58f4-4566-9789-0387a50da771 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:38.410539 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7eaaa-58f4-4566-9789-0387a50da771 None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:38.411093 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7eaaa-58f4-4566-9789-0387a50da771 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:38.411394 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7eaaa-58f4-4566-9789-0387a50da771 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:38.412766 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7eaaa-58f4-4566-9789-0387a50da771 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:38.425960 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 449/896] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:38.431123 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7eaaa-58f4-4566-9789-0387a50da771 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:38.463686 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:38.517763 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82015eb-0b04-416a-b913-5c35727af74d None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:38.541130 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7eaaa-58f4-4566-9789-0387a50da771 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:38.554824 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3faabb-a219-4306-be43-c27c1c7236b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:38.555794 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3faabb-a219-4306-be43-c27c1c7236b3 None admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:38.556298 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3faabb-a219-4306-be43-c27c1c7236b3 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:38.556744 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3faabb-a219-4306-be43-c27c1c7236b3 None admin] PATH_INFO: `/v3/services` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:38.557575 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3faabb-a219-4306-be43-c27c1c7236b3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:38.559267 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3faabb-a219-4306-be43-c27c1c7236b3 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:38.585359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c0c7f-49f8-4d56-8b31-751c9d0f9a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:38.585359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c0c7f-49f8-4d56-8b31-751c9d0f9a9e None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:38.585359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c0c7f-49f8-4d56-8b31-751c9d0f9a9e None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:38.585359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c0c7f-49f8-4d56-8b31-751c9d0f9a9e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:38.586045 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 448/897] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 10 22:35:38 2022] POST /identity/v3/services => generated 241 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:35:38.595641 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c0c7f-49f8-4d56-8b31-751c9d0f9a9e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:38.597451 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c0c7f-49f8-4d56-8b31-751c9d0f9a9e None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:38.612028 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-5f7c0c7f-49f8-4d56-8b31-751c9d0f9a9e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 22:35:38.612028 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:38.612028 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:38.612028 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:38.612028 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:38.612028 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:38.612028 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:38.612028 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:38.612028 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:38.612028 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:38.612028 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:38.612028 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:38.612028 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:35:38.612028 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:35:38.612028 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:35:38.612028 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:35:38.613247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:38.613247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:38.613247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:35:38.613247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:38.613247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:35:38.613247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:38.613247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:38.613247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:38.613247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:38.613247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:38.613247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:38.613247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:38.613247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:38.613247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:38.613247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:38.613247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:38.613247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:38.613247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:38.614244 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:38.614244 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:38.614244 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:35:38.614244 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:35:38.614244 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:35:38.614244 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:35:38.614244 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:35:38.614244 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:35:38.614244 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 22:35:38.614244 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:38.615052 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 450/898] 10.222.0.76 () {56 vars in 1117 bytes} [Wed Aug 10 22:35:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:35:38.620498 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3faabb-a219-4306-be43-c27c1c7236b3 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:38.741682 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3faabb-a219-4306-be43-c27c1c7236b3 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:38.752485 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:38.781815 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d59ef-ae27-4a76-b9b5-bce9aba7b456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:38.782438 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d59ef-ae27-4a76-b9b5-bce9aba7b456 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:38.782714 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d59ef-ae27-4a76-b9b5-bce9aba7b456 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:38.782984 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d59ef-ae27-4a76-b9b5-bce9aba7b456 None admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:38.783715 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d59ef-ae27-4a76-b9b5-bce9aba7b456 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:38.784018 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d59ef-ae27-4a76-b9b5-bce9aba7b456 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:38.785471 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d59ef-ae27-4a76-b9b5-bce9aba7b456 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:38.805102 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 449/899] 10.222.0.76 () {56 vars in 1149 bytes} [Wed Aug 10 22:35:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:35:38.811829 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c0c7f-49f8-4d56-8b31-751c9d0f9a9e None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:38.865303 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:38.924823 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c0c7f-49f8-4d56-8b31-751c9d0f9a9e None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:38.965866 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e67d3-c76a-4386-be3b-58aaf997808f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:38.966819 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e67d3-c76a-4386-be3b-58aaf997808f None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:38.967199 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e67d3-c76a-4386-be3b-58aaf997808f None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:38.968018 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e67d3-c76a-4386-be3b-58aaf997808f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:39.048376 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8e67d3-c76a-4386-be3b-58aaf997808f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:39.049783 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8e67d3-c76a-4386-be3b-58aaf997808f None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:39.096491 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:39.153960 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 451/900] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:35:38 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 22:35:39.159458 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d59ef-ae27-4a76-b9b5-bce9aba7b456 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:39.272299 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d59ef-ae27-4a76-b9b5-bce9aba7b456 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:39.301225 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e28c37-de37-4499-bdb0-d6e5372591de None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:39.301551 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e28c37-de37-4499-bdb0-d6e5372591de None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:39.301745 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e28c37-de37-4499-bdb0-d6e5372591de None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:39.303207 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 452/901] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:35:39.310215 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d276f5-6337-4463-bb91-0c65b895136d None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:39.310469 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d276f5-6337-4463-bb91-0c65b895136d None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:39.310690 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d276f5-6337-4463-bb91-0c65b895136d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:39.314968 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2243333-7d91-4ed8-992e-9dccb5e6f1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:39.315866 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2243333-7d91-4ed8-992e-9dccb5e6f1b8 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:39.316350 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2243333-7d91-4ed8-992e-9dccb5e6f1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:39.316824 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2243333-7d91-4ed8-992e-9dccb5e6f1b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:39.329454 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2243333-7d91-4ed8-992e-9dccb5e6f1b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:39.331615 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2243333-7d91-4ed8-992e-9dccb5e6f1b8 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:39.337669 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-e2d276f5-6337-4463-bb91-0c65b895136d None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:39.339023 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-e2d276f5-6337-4463-bb91-0c65b895136d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:39.350122 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-c2243333-7d91-4ed8-992e-9dccb5e6f1b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:39.350122 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:39.350122 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:39.350122 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:39.350122 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:39.350122 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:39.350122 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:39.350122 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:39.350122 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:39.350122 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:39.350122 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:39.350122 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:39.350122 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:39.350122 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:39.350122 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:39.350122 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:39.351133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:39.351133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:39.351133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:39.351133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:39.351133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:39.351133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:39.351133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:39.351133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:39.351133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:39.351133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:39.351133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:39.351133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:39.351133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:39.351133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:39.351133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:39.351133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:39.351133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:39.352039 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:39.352039 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:39.352039 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:39.353374 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 450/902] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:39.357380 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2243333-7d91-4ed8-992e-9dccb5e6f1b8 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:39.385406 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:39.457306 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 453/903] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:35:39.475167 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d276f5-6337-4463-bb91-0c65b895136d None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:39.510473 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2243333-7d91-4ed8-992e-9dccb5e6f1b8 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:39.570849 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c68cc3-805d-4753-9971-1b2e07ee4c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:39.572402 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c68cc3-805d-4753-9971-1b2e07ee4c64 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:39.573154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c68cc3-805d-4753-9971-1b2e07ee4c64 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:39.574278 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c68cc3-805d-4753-9971-1b2e07ee4c64 None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:39.576435 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c68cc3-805d-4753-9971-1b2e07ee4c64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:39.577383 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c68cc3-805d-4753-9971-1b2e07ee4c64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:39.580696 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c68cc3-805d-4753-9971-1b2e07ee4c64 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:39.601270 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:39.602613 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 451/904] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:35:39.609836 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c68cc3-805d-4753-9971-1b2e07ee4c64 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:39.641024 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d276f5-6337-4463-bb91-0c65b895136d None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:39.686304 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b7224-3e3e-461a-b093-e29fc2b61671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:39.687377 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b7224-3e3e-461a-b093-e29fc2b61671 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:39.687863 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b7224-3e3e-461a-b093-e29fc2b61671 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:39.688359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b7224-3e3e-461a-b093-e29fc2b61671 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:39.689144 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b7224-3e3e-461a-b093-e29fc2b61671 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:39.690755 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b7224-3e3e-461a-b093-e29fc2b61671 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:39.700767 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-448b7224-3e3e-461a-b093-e29fc2b61671 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 22:35:39.700767 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:39.700767 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:35:39.700767 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:35:39.700767 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:35:39.700767 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:35:39.700767 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:35:39.700767 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:39.700767 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 22:35:39.700767 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:39.700767 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:35:39.700767 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:39.700767 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:39.700767 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:39.700767 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:39.700767 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:39.701994 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:39.701994 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:39.701994 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:39.701994 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:39.701994 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:39.701994 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:39.701994 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:39.701994 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:35:39.701994 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:35:39.701994 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:35:39.701994 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:35:39.701994 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:39.701994 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:39.701994 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:39.701994 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:39.701994 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:39.701994 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:39.703045 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:39.703045 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:39.703045 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:39.703045 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:39.703045 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:39.703045 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:39.703045 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:39.703045 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:39.703045 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:39.703045 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:39.703045 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:35:39.703045 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:39.703045 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 22:35:39.703045 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:39.705912 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 454/905] 10.222.0.76 () {56 vars in 1132 bytes} [Wed Aug 10 22:35:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:35:39.711994 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b7224-3e3e-461a-b093-e29fc2b61671 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:39.724832 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:39.750822 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c68cc3-805d-4753-9971-1b2e07ee4c64 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:39.787976 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd0e59-8cdd-4167-bd4c-c0370a342d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:39.788618 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd0e59-8cdd-4167-bd4c-c0370a342d0e None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:39.788897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd0e59-8cdd-4167-bd4c-c0370a342d0e None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:39.789172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd0e59-8cdd-4167-bd4c-c0370a342d0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:39.799250 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd0e59-8cdd-4167-bd4c-c0370a342d0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:39.800792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd0e59-8cdd-4167-bd4c-c0370a342d0e None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:39.812316 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-99cd0e59-8cdd-4167-bd4c-c0370a342d0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:39.812316 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:39.812316 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:39.812316 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:39.812316 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:39.812316 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:39.812316 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:39.812316 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:39.812316 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:39.812316 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:39.812316 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:39.812316 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:39.812316 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:35:39.812316 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:35:39.812316 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:35:39.813246 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:35:39.813246 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:39.813246 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:39.813246 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:39.813246 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:39.813246 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:39.813246 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:39.813246 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:39.813246 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:39.813246 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:39.813246 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:39.813246 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:39.813246 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:39.813246 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:39.813246 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:39.813246 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:39.813246 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:39.815062 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:35:39.815062 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:35:39.815062 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:35:39.815062 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:35:39.815062 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:35:39.815062 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:35:39.815062 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:39.815062 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:39.815062 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 452/906] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:35:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:35:39.815062 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e93fe-4ab6-49a1-a2a4-fde175109ae8 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:39.815627 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e93fe-4ab6-49a1-a2a4-fde175109ae8 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:39.815627 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e93fe-4ab6-49a1-a2a4-fde175109ae8 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:39.816863 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 453/907] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:39.819612 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e93fe-4ab6-49a1-a2a4-fde175109ae8 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:39.831643 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b7224-3e3e-461a-b093-e29fc2b61671 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:39.867203 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b37fa5d-9ed8-41d9-aba6-daca851fb423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:39.867812 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37fa5d-9ed8-41d9-aba6-daca851fb423 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:39.868092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37fa5d-9ed8-41d9-aba6-daca851fb423 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:39.868364 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37fa5d-9ed8-41d9-aba6-daca851fb423 None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:39.868993 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b37fa5d-9ed8-41d9-aba6-daca851fb423 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:39.869307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b37fa5d-9ed8-41d9-aba6-daca851fb423 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:39.870689 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b37fa5d-9ed8-41d9-aba6-daca851fb423 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:39.880786 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 455/908] 10.222.0.76 () {56 vars in 1126 bytes} [Wed Aug 10 22:35:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:35:39.880786 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfbf3b-1188-4e71-9373-5e1d8d14fac6 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:39.880786 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfbf3b-1188-4e71-9373-5e1d8d14fac6 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:39.880786 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfbf3b-1188-4e71-9373-5e1d8d14fac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:39.884837 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:39.899512 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-7fcfbf3b-1188-4e71-9373-5e1d8d14fac6 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:39.900688 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-7fcfbf3b-1188-4e71-9373-5e1d8d14fac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:39.943790 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e93fe-4ab6-49a1-a2a4-fde175109ae8 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:39.981306 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 456/909] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:35:39.981306 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfbf3b-1188-4e71-9373-5e1d8d14fac6 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:39.992683 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5578b4-c7a1-4f3f-8ac6-bc16af3347d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:39.993783 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5578b4-c7a1-4f3f-8ac6-bc16af3347d0 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:39.994069 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5578b4-c7a1-4f3f-8ac6-bc16af3347d0 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:39.994345 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5578b4-c7a1-4f3f-8ac6-bc16af3347d0 None admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:39.995057 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5578b4-c7a1-4f3f-8ac6-bc16af3347d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:39.995363 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5578b4-c7a1-4f3f-8ac6-bc16af3347d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:39.997623 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5578b4-c7a1-4f3f-8ac6-bc16af3347d0 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:40.006779 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:40.018173 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 454/910] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:35:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:40.019081 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5578b4-c7a1-4f3f-8ac6-bc16af3347d0 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:40.092096 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfbf3b-1188-4e71-9373-5e1d8d14fac6 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:40.123919 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:40.129604 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5578b4-c7a1-4f3f-8ac6-bc16af3347d0 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:40.130404 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2a662-6945-410b-817f-0a2351529eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:40.131181 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2a662-6945-410b-817f-0a2351529eb1 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:40.131466 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2a662-6945-410b-817f-0a2351529eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:40.132507 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2a662-6945-410b-817f-0a2351529eb1 None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:40.133159 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2a662-6945-410b-817f-0a2351529eb1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:40.133527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2a662-6945-410b-817f-0a2351529eb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:40.134983 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2a662-6945-410b-817f-0a2351529eb1 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:40.146727 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 457/911] 10.222.0.76 () {56 vars in 1126 bytes} [Wed Aug 10 22:35:39 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:40.146727 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2a662-6945-410b-817f-0a2351529eb1 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:40.164483 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6fc154-86b6-4eb2-9fca-335feaa0fcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:40.165112 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6fc154-86b6-4eb2-9fca-335feaa0fcac None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:40.165416 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6fc154-86b6-4eb2-9fca-335feaa0fcac None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:40.165690 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6fc154-86b6-4eb2-9fca-335feaa0fcac None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:40.166257 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6fc154-86b6-4eb2-9fca-335feaa0fcac None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:40.167606 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6fc154-86b6-4eb2-9fca-335feaa0fcac None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:40.178147 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-6e6fc154-86b6-4eb2-9fca-335feaa0fcac None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 22:35:40.178147 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:40.178147 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:35:40.178147 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:35:40.178147 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:35:40.178147 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:35:40.178147 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:35:40.178147 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:40.178147 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 22:35:40.178147 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:40.178147 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:35:40.178147 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:40.178147 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:40.178147 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:40.178147 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:40.178147 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:40.178147 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:40.179172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:40.179172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:40.179172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:40.179172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:40.179172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:40.179172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:40.179172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:35:40.179172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:35:40.179172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:35:40.179172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:35:40.179172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:40.179172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:40.179172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:40.179172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:40.179172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:40.179172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:40.180028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:40.180028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:40.180028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:40.180028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:40.180028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:40.180028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:40.180028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:40.180028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:40.180028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:40.180028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:40.180028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:35:40.180028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:40.180028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 22:35:40.180028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:40.180028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 455/912] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 10 22:35:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:40.180840 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6fc154-86b6-4eb2-9fca-335feaa0fcac None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:40.245067 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:40.269623 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2a662-6945-410b-817f-0a2351529eb1 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:40.288819 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6fc154-86b6-4eb2-9fca-335feaa0fcac None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:40.304583 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6addd-b106-4b4b-927f-0172688857df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:40.305191 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6addd-b106-4b4b-927f-0172688857df None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:40.305616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6addd-b106-4b4b-927f-0172688857df None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:40.305779 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6addd-b106-4b4b-927f-0172688857df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:40.306506 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f6addd-b106-4b4b-927f-0172688857df None admin] RBAC: Adding query filter params (role.id=ee973ce49dc142a6b0763794e078333d, scope.project.id=3caa2ea846f7449b8c483788fc5846c8, user.id=aff5e804c64345b298c745287a1d0a32) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:40.306812 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f6addd-b106-4b4b-927f-0172688857df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:40.308249 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f6addd-b106-4b4b-927f-0172688857df None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:40.313681 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 458/913] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:35:40 2022] GET /identity/v3/role_assignments?user.id=aff5e804c64345b298c745287a1d0a32&scope.project.id=3caa2ea846f7449b8c483788fc5846c8&role.id=ee973ce49dc142a6b0763794e078333d => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:40.314476 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6addd-b106-4b4b-927f-0172688857df None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:40.336297 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8fdafb-5c01-487a-9ca4-cf5af54b0e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:40.337467 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fdafb-5c01-487a-9ca4-cf5af54b0e53 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:40.338025 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fdafb-5c01-487a-9ca4-cf5af54b0e53 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:40.338486 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fdafb-5c01-487a-9ca4-cf5af54b0e53 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:40.339518 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8fdafb-5c01-487a-9ca4-cf5af54b0e53 None admin] RBAC: Adding query filter params (interface=public, service_id=0af8c9ca5af54bc6ac50993bba11bb58, region_id=RegionOne) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:40.341109 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8fdafb-5c01-487a-9ca4-cf5af54b0e53 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:40.343006 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8fdafb-5c01-487a-9ca4-cf5af54b0e53 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:40.349546 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 456/914] 10.222.0.76 () {56 vars in 1251 bytes} [Wed Aug 10 22:35:40 2022] GET /identity/v3/endpoints?service_id=0af8c9ca5af54bc6ac50993bba11bb58&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:40.356135 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8fdafb-5c01-487a-9ca4-cf5af54b0e53 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:40.401819 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:40.473509 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6addd-b106-4b4b-927f-0172688857df None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:40.498716 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8fdafb-5c01-487a-9ca4-cf5af54b0e53 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:40.512855 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b1d79c-ab46-4709-9ef4-0a7cf5c15d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:40.513859 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1d79c-ab46-4709-9ef4-0a7cf5c15d45 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:40.514356 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1d79c-ab46-4709-9ef4-0a7cf5c15d45 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:40.514730 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1d79c-ab46-4709-9ef4-0a7cf5c15d45 None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:40.515482 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b1d79c-ab46-4709-9ef4-0a7cf5c15d45 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:40.515887 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b1d79c-ab46-4709-9ef4-0a7cf5c15d45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:40.517392 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b1d79c-ab46-4709-9ef4-0a7cf5c15d45 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:40.527013 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 459/915] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 10 22:35:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:40.529257 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b1d79c-ab46-4709-9ef4-0a7cf5c15d45 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:40.531352 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:40.541659 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae0294-cf2b-40bd-a22c-9ad62902c886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:40.543423 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae0294-cf2b-40bd-a22c-9ad62902c886 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:40.543908 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae0294-cf2b-40bd-a22c-9ad62902c886 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:40.544401 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae0294-cf2b-40bd-a22c-9ad62902c886 None admin] PATH_INFO: `/v3/services` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:40.545219 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae0294-cf2b-40bd-a22c-9ad62902c886 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:40.547018 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae0294-cf2b-40bd-a22c-9ad62902c886 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:40.554441 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 457/916] 10.222.0.76 () {56 vars in 1087 bytes} [Wed Aug 10 22:35:40 2022] GET /identity/v3/services => generated 1497 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 22:35:40.650024 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b1d79c-ab46-4709-9ef4-0a7cf5c15d45 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:40.678686 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:40.691798 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8db51-9207-4b0d-ae07-79d8464f9c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:40.692726 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8db51-9207-4b0d-ae07-79d8464f9c13 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:40.693009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8db51-9207-4b0d-ae07-79d8464f9c13 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:40.693372 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8db51-9207-4b0d-ae07-79d8464f9c13 None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:40.694015 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8db51-9207-4b0d-ae07-79d8464f9c13 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:40.694324 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8db51-9207-4b0d-ae07-79d8464f9c13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:40.695724 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8db51-9207-4b0d-ae07-79d8464f9c13 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:40.702585 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 460/917] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 10 22:35:40 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:40.710654 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae0294-cf2b-40bd-a22c-9ad62902c886 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:40.824822 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae0294-cf2b-40bd-a22c-9ad62902c886 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:40.863487 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0549fe96-5260-49aa-8a0f-0d6fdad6024f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:40.864101 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549fe96-5260-49aa-8a0f-0d6fdad6024f None admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:40.864422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549fe96-5260-49aa-8a0f-0d6fdad6024f None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:40.864703 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549fe96-5260-49aa-8a0f-0d6fdad6024f None admin] PATH_INFO: `/v3/endpoints` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:40.865395 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0549fe96-5260-49aa-8a0f-0d6fdad6024f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:40.866758 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0549fe96-5260-49aa-8a0f-0d6fdad6024f None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:40.897441 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 458/918] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:35:40 2022] POST /identity/v3/endpoints => generated 346 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:35:40.919766 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86689565-32a9-4cf1-a04a-dc38c15f2662 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:40.919977 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86689565-32a9-4cf1-a04a-dc38c15f2662 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:40.920233 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86689565-32a9-4cf1-a04a-dc38c15f2662 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:40.921981 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 461/919] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:40.927945 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12423165-5bdc-4fd0-b95a-9b4fd1395a11 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:40.928282 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12423165-5bdc-4fd0-b95a-9b4fd1395a11 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:40.928510 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12423165-5bdc-4fd0-b95a-9b4fd1395a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:40.948546 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-12423165-5bdc-4fd0-b95a-9b4fd1395a11 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:40.950353 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-12423165-5bdc-4fd0-b95a-9b4fd1395a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:41.002722 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:41.038056 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 459/920] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:35:41.063521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86689565-32a9-4cf1-a04a-dc38c15f2662 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:41.186260 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86689565-32a9-4cf1-a04a-dc38c15f2662 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:41.226605 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf14e8b7-ef32-4f54-bcd6-3007f884ecc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:41.227252 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf14e8b7-ef32-4f54-bcd6-3007f884ecc6 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:41.227536 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf14e8b7-ef32-4f54-bcd6-3007f884ecc6 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:41.227812 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf14e8b7-ef32-4f54-bcd6-3007f884ecc6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:41.229260 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf14e8b7-ef32-4f54-bcd6-3007f884ecc6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:41.230727 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf14e8b7-ef32-4f54-bcd6-3007f884ecc6 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:41.242228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-cf14e8b7-ef32-4f54-bcd6-3007f884ecc6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 22:35:41.242228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:41.242228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:35:41.242228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:35:41.242228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:35:41.242228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:35:41.242228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:35:41.242228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:41.242228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 22:35:41.242228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:41.242228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:35:41.242228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:41.242228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:41.242228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:41.242228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:41.242228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:41.242228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:41.243294 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:41.243294 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:41.243294 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:41.243294 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:41.243294 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:41.243294 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:41.243294 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:35:41.243294 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:35:41.243294 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:35:41.243294 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:35:41.243294 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:41.243294 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:41.243294 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:41.243294 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:41.243294 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:41.243294 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:41.244187 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:41.244187 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:41.244187 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:41.244187 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:41.244187 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:41.244187 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:41.244187 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:41.244187 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:41.244187 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:41.244187 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:41.244187 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:35:41.244187 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:41.244187 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 22:35:41.244187 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:41.247796 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 462/921] 10.222.0.76 () {56 vars in 1126 bytes} [Wed Aug 10 22:35:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:41.253521 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12423165-5bdc-4fd0-b95a-9b4fd1395a11 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:41.447507 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12423165-5bdc-4fd0-b95a-9b4fd1395a11 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:41.510634 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee06604-718b-4dfc-b40b-baa0846dee33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:41.511571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee06604-718b-4dfc-b40b-baa0846dee33 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:41.517318 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee06604-718b-4dfc-b40b-baa0846dee33 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:41.517318 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee06604-718b-4dfc-b40b-baa0846dee33 None admin] PATH_INFO: `/v3/services` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:41.517318 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee06604-718b-4dfc-b40b-baa0846dee33 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:41.517318 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee06604-718b-4dfc-b40b-baa0846dee33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:41.522448 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee06604-718b-4dfc-b40b-baa0846dee33 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:41.533865 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 460/922] 10.222.0.76 () {56 vars in 1122 bytes} [Wed Aug 10 22:35:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:41.541881 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf14e8b7-ef32-4f54-bcd6-3007f884ecc6 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:41.641968 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:41.715603 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf14e8b7-ef32-4f54-bcd6-3007f884ecc6 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:41.751817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d0c876-f9bf-4466-9d3c-fd5e58ccb3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:41.752430 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0c876-f9bf-4466-9d3c-fd5e58ccb3e9 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:41.752711 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0c876-f9bf-4466-9d3c-fd5e58ccb3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:41.752987 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0c876-f9bf-4466-9d3c-fd5e58ccb3e9 None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:41.753648 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d0c876-f9bf-4466-9d3c-fd5e58ccb3e9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:41.753949 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d0c876-f9bf-4466-9d3c-fd5e58ccb3e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:41.755385 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d0c876-f9bf-4466-9d3c-fd5e58ccb3e9 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:41.760818 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 463/923] 10.222.0.76 () {56 vars in 1122 bytes} [Wed Aug 10 22:35:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:41.767516 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee06604-718b-4dfc-b40b-baa0846dee33 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:41.806032 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:41.888147 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee06604-718b-4dfc-b40b-baa0846dee33 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:41.927711 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f4873-441b-435e-be36-d2651e36703f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:41.928328 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f4873-441b-435e-be36-d2651e36703f None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:41.928608 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f4873-441b-435e-be36-d2651e36703f None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:41.928882 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f4873-441b-435e-be36-d2651e36703f None admin] PATH_INFO: `/v3/endpoints` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:41.929611 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848f4873-441b-435e-be36-d2651e36703f None admin] RBAC: Adding query filter params (interface=public, service_id=a74afca16248445e951e92a5dccfa41b, region_id=RegionOne) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:41.929914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848f4873-441b-435e-be36-d2651e36703f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:41.931286 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848f4873-441b-435e-be36-d2651e36703f None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:41.937706 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 461/924] 10.222.0.76 () {56 vars in 1251 bytes} [Wed Aug 10 22:35:41 2022] GET /identity/v3/endpoints?service_id=a74afca16248445e951e92a5dccfa41b&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:41.943205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d0c876-f9bf-4466-9d3c-fd5e58ccb3e9 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:42.051038 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d0c876-f9bf-4466-9d3c-fd5e58ccb3e9 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:42.068234 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:42.087417 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a8efaa-4b06-4e6b-8f14-eadd3c1f8d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:42.088032 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8efaa-4b06-4e6b-8f14-eadd3c1f8d2e None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:42.088314 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8efaa-4b06-4e6b-8f14-eadd3c1f8d2e None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:42.088587 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8efaa-4b06-4e6b-8f14-eadd3c1f8d2e None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:42.089789 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a8efaa-4b06-4e6b-8f14-eadd3c1f8d2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:42.091179 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a8efaa-4b06-4e6b-8f14-eadd3c1f8d2e None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:42.101201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 464/925] 10.222.0.76 () {56 vars in 1087 bytes} [Wed Aug 10 22:35:41 2022] GET /identity/v3/services => generated 1497 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:35:42.545818 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c879b-f8dd-4c46-bf1d-0e26cb9af0bc None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:42.546523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c879b-f8dd-4c46-bf1d-0e26cb9af0bc None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:42.546523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c879b-f8dd-4c46-bf1d-0e26cb9af0bc None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:42.548056 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 462/926] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:42.553391 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623ecbc-8fb6-4a50-bf88-b1b732430366 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:42.553684 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623ecbc-8fb6-4a50-bf88-b1b732430366 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:42.553965 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623ecbc-8fb6-4a50-bf88-b1b732430366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:42.576545 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-6623ecbc-8fb6-4a50-bf88-b1b732430366 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:42.576545 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-6623ecbc-8fb6-4a50-bf88-b1b732430366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:42.648226 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 465/927] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:35:42.671823 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c879b-f8dd-4c46-bf1d-0e26cb9af0bc None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:42.770449 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e436f-5d79-47a7-9a48-f62f9ea8f3a4 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:42.771071 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e436f-5d79-47a7-9a48-f62f9ea8f3a4 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:42.771559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e436f-5d79-47a7-9a48-f62f9ea8f3a4 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:42.773426 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 466/928] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:42.779104 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa82b9-84e0-447f-9157-0b55c1b4180a None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:42.779527 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:42.779833 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa82b9-84e0-447f-9157-0b55c1b4180a None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:42.780326 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa82b9-84e0-447f-9157-0b55c1b4180a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:42.783638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c879b-f8dd-4c46-bf1d-0e26cb9af0bc None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:42.800756 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-7dfa82b9-84e0-447f-9157-0b55c1b4180a None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:42.802459 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-7dfa82b9-84e0-447f-9157-0b55c1b4180a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:42.821530 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e3604c-7176-4c93-96e7-bce62bfaf946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:42.822177 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3604c-7176-4c93-96e7-bce62bfaf946 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:42.822459 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3604c-7176-4c93-96e7-bce62bfaf946 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:42.822730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3604c-7176-4c93-96e7-bce62bfaf946 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:42.834424 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e3604c-7176-4c93-96e7-bce62bfaf946 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:35:42.840522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-a2e3604c-7176-4c93-96e7-bce62bfaf946 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 22:35:42.840522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:42.840522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:42.840522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:42.840522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:42.840522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:42.840522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:42.840522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:42.840522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:42.840522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:42.840522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:42.840522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:42.840522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:35:42.840522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:35:42.840522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:35:42.840522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:35:42.841766 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:35:42.841766 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:35:42.841766 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:35:42.841766 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:35:42.841766 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:42.841766 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:42.841766 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:42.841766 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:42.841766 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:42.841766 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:42.841766 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:42.841766 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:42.841766 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:42.841766 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:42.841766 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:42.841766 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:42.842607 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:42.842607 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:42.842607 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:42.842607 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:42.842607 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:35:42.842607 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:35:42.842607 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:35:42.842607 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:35:42.842607 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:35:42.842607 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:35:42.842607 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 22:35:42.842607 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:42.842607 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 463/929] 10.222.0.76 () {56 vars in 1120 bytes} [Wed Aug 10 22:35:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:42.847493 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e3604c-7176-4c93-96e7-bce62bfaf946 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:42.892842 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 467/930] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:35:42.911905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa82b9-84e0-447f-9157-0b55c1b4180a None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:42.958568 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:42.981954 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e3604c-7176-4c93-96e7-bce62bfaf946 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:43.025581 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f763753-c87e-4300-a6f7-74caf48134b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:43.026939 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f763753-c87e-4300-a6f7-74caf48134b6 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:43.027220 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f763753-c87e-4300-a6f7-74caf48134b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:43.027494 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f763753-c87e-4300-a6f7-74caf48134b6 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:43.028211 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f763753-c87e-4300-a6f7-74caf48134b6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:43.028525 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f763753-c87e-4300-a6f7-74caf48134b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:43.030638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f763753-c87e-4300-a6f7-74caf48134b6 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:43.032186 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa82b9-84e0-447f-9157-0b55c1b4180a None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:43.040490 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 464/931] 10.222.0.76 () {56 vars in 1115 bytes} [Wed Aug 10 22:35:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:43.045842 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f763753-c87e-4300-a6f7-74caf48134b6 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:43.075927 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b6346-42ff-40ce-86cf-e138171c7093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:43.077183 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b6346-42ff-40ce-86cf-e138171c7093 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:43.077651 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b6346-42ff-40ce-86cf-e138171c7093 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:43.078091 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b6346-42ff-40ce-86cf-e138171c7093 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:43.079293 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b6346-42ff-40ce-86cf-e138171c7093 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:43.081102 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b6346-42ff-40ce-86cf-e138171c7093 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:43.081641 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:43.094060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-6d9b6346-42ff-40ce-86cf-e138171c7093 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 22:35:43.094060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:43.094060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:35:43.094060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:35:43.094060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:35:43.094060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:35:43.094060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:35:43.094060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:43.094060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 22:35:43.094060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:43.094060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:35:43.094060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:43.094060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:43.094060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:43.094060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:43.094060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:43.095276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:43.095276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:43.095276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:43.095276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:43.095276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:43.095276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:43.095276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:43.095276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:35:43.095276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:35:43.095276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:35:43.095276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:35:43.095276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:43.095276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:43.095276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:43.095276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:43.095276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:43.095276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:43.096374 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:43.096374 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:43.096374 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:43.096374 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:43.096374 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:43.096374 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:43.096374 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:43.096374 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:43.096374 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:43.096374 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:43.096374 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:35:43.096374 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:43.096374 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 22:35:43.096374 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:43.101353 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 468/932] 10.222.0.76 () {56 vars in 1132 bytes} [Wed Aug 10 22:35:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:35:43.105517 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b6346-42ff-40ce-86cf-e138171c7093 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:43.175644 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f763753-c87e-4300-a6f7-74caf48134b6 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:43.220060 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cabde4c-8a15-4459-9819-7b42e9544711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:43.221569 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabde4c-8a15-4459-9819-7b42e9544711 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:43.222023 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabde4c-8a15-4459-9819-7b42e9544711 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:43.222459 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabde4c-8a15-4459-9819-7b42e9544711 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:43.235620 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b6346-42ff-40ce-86cf-e138171c7093 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:43.238382 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:43.240404 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cabde4c-8a15-4459-9819-7b42e9544711 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:43.242125 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cabde4c-8a15-4459-9819-7b42e9544711 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:43.259866 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-7cabde4c-8a15-4459-9819-7b42e9544711 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:43.259866 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:43.259866 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:43.259866 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:43.259866 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:43.259866 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:43.259866 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:43.259866 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:43.259866 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:43.259866 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:43.259866 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:43.259866 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:43.259866 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:43.259866 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:43.259866 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:43.259866 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:43.260959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:43.260959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:43.260959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:43.260959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:43.260959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:43.260959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:43.260959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:43.260959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:43.260959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:43.260959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:43.260959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:43.260959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:43.260959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:43.260959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:43.260959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:43.260959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:43.260959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:43.261928 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:43.261928 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:43.261928 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:43.265930 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 465/933] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:35:43.267116 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41ff34-dea9-4778-bf87-5f6dbe941c9c None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:43.267404 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41ff34-dea9-4778-bf87-5f6dbe941c9c None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:43.267687 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41ff34-dea9-4778-bf87-5f6dbe941c9c None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:43.269084 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 466/934] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:43.269657 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41ff34-dea9-4778-bf87-5f6dbe941c9c None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:43.281306 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95719a4f-a7dd-4d8f-a103-9725d8e49f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:43.281908 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95719a4f-a7dd-4d8f-a103-9725d8e49f46 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:43.282127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95719a4f-a7dd-4d8f-a103-9725d8e49f46 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:43.282396 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95719a4f-a7dd-4d8f-a103-9725d8e49f46 None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:43.283258 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95719a4f-a7dd-4d8f-a103-9725d8e49f46 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:43.283363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95719a4f-a7dd-4d8f-a103-9725d8e49f46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:43.284681 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95719a4f-a7dd-4d8f-a103-9725d8e49f46 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:43.289617 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 469/935] 10.222.0.76 () {56 vars in 1126 bytes} [Wed Aug 10 22:35:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:43.289617 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746016df-31e8-4155-93aa-d526e31a5ec6 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:43.289617 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746016df-31e8-4155-93aa-d526e31a5ec6 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:43.289617 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746016df-31e8-4155-93aa-d526e31a5ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:43.311165 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-746016df-31e8-4155-93aa-d526e31a5ec6 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:43.312390 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-746016df-31e8-4155-93aa-d526e31a5ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:43.362286 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:43.383878 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 470/936] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:35:43.384619 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746016df-31e8-4155-93aa-d526e31a5ec6 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:43.395531 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41ff34-dea9-4778-bf87-5f6dbe941c9c None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:43.438303 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dde6a9-79d7-4488-8632-a5efc7af1b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:43.439106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dde6a9-79d7-4488-8632-a5efc7af1b57 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:43.439388 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dde6a9-79d7-4488-8632-a5efc7af1b57 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:43.439661 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dde6a9-79d7-4488-8632-a5efc7af1b57 None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:43.440301 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dde6a9-79d7-4488-8632-a5efc7af1b57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:43.440602 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dde6a9-79d7-4488-8632-a5efc7af1b57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:43.442264 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dde6a9-79d7-4488-8632-a5efc7af1b57 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:43.454832 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 467/937] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:43.455612 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dde6a9-79d7-4488-8632-a5efc7af1b57 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:43.506709 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:43.511665 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746016df-31e8-4155-93aa-d526e31a5ec6 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:43.553186 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304fb8f-abe9-42c4-8e6e-766f319235d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:43.553745 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304fb8f-abe9-42c4-8e6e-766f319235d0 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:43.554022 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304fb8f-abe9-42c4-8e6e-766f319235d0 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:43.554291 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304fb8f-abe9-42c4-8e6e-766f319235d0 None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:43.554916 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d304fb8f-abe9-42c4-8e6e-766f319235d0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:43.555211 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d304fb8f-abe9-42c4-8e6e-766f319235d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:43.556602 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d304fb8f-abe9-42c4-8e6e-766f319235d0 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:43.565435 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 471/938] 10.222.0.76 () {56 vars in 1126 bytes} [Wed Aug 10 22:35:43 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:43.565435 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304fb8f-abe9-42c4-8e6e-766f319235d0 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:43.571344 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dde6a9-79d7-4488-8632-a5efc7af1b57 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:43.609868 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd170e1-0c29-4e75-8e73-0eb3433edca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:43.610581 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd170e1-0c29-4e75-8e73-0eb3433edca0 None admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:43.610870 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd170e1-0c29-4e75-8e73-0eb3433edca0 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:43.611173 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd170e1-0c29-4e75-8e73-0eb3433edca0 None admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:43.611817 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd170e1-0c29-4e75-8e73-0eb3433edca0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:43.613250 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd170e1-0c29-4e75-8e73-0eb3433edca0 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:43.615882 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: WARNING keystone.server.flask.common [None req-cfd170e1-0c29-4e75-8e73-0eb3433edca0 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 10 22:35:43.619414 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:43.635343 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.sql.core [None req-cfd170e1-0c29-4e75-8e73-0eb3433edca0 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 22:35:43.635343 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 10 22:35:43.635343 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [parameters: {'id': 'd93df2e0915d45b6b83e26afff7d75de', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 10 22:35:43.635343 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93439) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 22:35:43.643254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-cfd170e1-0c29-4e75-8e73-0eb3433edca0 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d93df2e0915d45b6b83e26afff7d75de.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d93df2e0915d45b6b83e26afff7d75de. Aug 10 22:35:43.643254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:43.643254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 22:35:43.643254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 22:35:43.643254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 22:35:43.643254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 22:35:43.643254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 22:35:43.643254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 22:35:43.643254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 22:35:43.643254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application conn.query(q) Aug 10 22:35:43.643254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 22:35:43.643254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 22:35:43.643254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 22:35:43.644094 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application result.read() Aug 10 22:35:43.644094 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 22:35:43.644094 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 22:35:43.644094 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 22:35:43.644094 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 22:35:43.644094 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 22:35:43.644094 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 22:35:43.644094 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 22:35:43.644094 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 22:35:43.644094 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 22:35:43.644094 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:43.644094 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 22:35:43.644094 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:43.644094 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:43.644094 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 22:35:43.644094 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 22:35:43.644094 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 10 22:35:43.644094 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 10 22:35:43.650652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 22:35:43.650652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application next(self.gen) Aug 10 22:35:43.650652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 22:35:43.650652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application yield resource Aug 10 22:35:43.650652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 22:35:43.650652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application next(self.gen) Aug 10 22:35:43.650652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 22:35:43.650652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 22:35:43.650652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 22:35:43.650652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 22:35:43.650652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 22:35:43.650652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise self.value Aug 10 22:35:43.650652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 22:35:43.650652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 22:35:43.650652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 22:35:43.650652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application session.commit() Aug 10 22:35:43.650652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 22:35:43.652195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 22:35:43.652195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 22:35:43.652195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 22:35:43.652195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 22:35:43.652195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self.session.flush() Aug 10 22:35:43.652195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 22:35:43.652195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 22:35:43.652195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 22:35:43.652195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 22:35:43.652195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 22:35:43.652195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application compat.raise_( Aug 10 22:35:43.652195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 22:35:43.652195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception Aug 10 22:35:43.652195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 22:35:43.652195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 22:35:43.652195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 22:35:43.652195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 22:35:43.653242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 22:35:43.653242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 22:35:43.653242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 22:35:43.653242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 22:35:43.653242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 10 22:35:43.653242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 10 22:35:43.653242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 22:35:43.653242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 22:35:43.653242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 22:35:43.653242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 22:35:43.653242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 22:35:43.653242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 22:35:43.653242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 22:35:43.653242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 22:35:43.653242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 22:35:43.654083 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 22:35:43.654083 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 22:35:43.654083 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception Aug 10 22:35:43.654083 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 22:35:43.654083 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 22:35:43.654083 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 22:35:43.654083 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 22:35:43.654083 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 22:35:43.654083 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 22:35:43.654083 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 22:35:43.654083 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application conn.query(q) Aug 10 22:35:43.654083 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 22:35:43.654083 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 22:35:43.654083 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 22:35:43.654083 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application result.read() Aug 10 22:35:43.654083 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 22:35:43.654083 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 22:35:43.657621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 22:35:43.657621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 22:35:43.657621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 22:35:43.657621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 22:35:43.657621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 22:35:43.657621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 22:35:43.657621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 22:35:43.657621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 10 22:35:43.657621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [parameters: {'id': 'd93df2e0915d45b6b83e26afff7d75de', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 10 22:35:43.657621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 22:35:43.657621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:43.657621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:35:43.657621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:43.657621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:43.657621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 10 22:35:43.659674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 10 22:35:43.659674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 22:35:43.659674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 22:35:43.659674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 10 22:35:43.659674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:43.659674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:35:43.659674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:43.659674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:43.659674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:43.659674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:43.659674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:43.659674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:43.659674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:43.659674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:43.659674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:43.659674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:43.659674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:43.661762 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:43.661762 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 10 22:35:43.661762 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 10 22:35:43.661762 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:43.661762 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:43.661762 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 10 22:35:43.661762 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 10 22:35:43.661762 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d93df2e0915d45b6b83e26afff7d75de. Aug 10 22:35:43.661762 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:43.661762 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 468/939] 10.222.0.76 () {60 vars in 1140 bytes} [Wed Aug 10 22:35:43 2022] POST /identity/v3/projects => generated 262 bytes in 189 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:43.661762 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd170e1-0c29-4e75-8e73-0eb3433edca0 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:43.684553 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304fb8f-abe9-42c4-8e6e-766f319235d0 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:43.723717 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9e41d-48d6-4c73-9055-92165a7bd691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:43.724354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9e41d-48d6-4c73-9055-92165a7bd691 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:43.724652 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9e41d-48d6-4c73-9055-92165a7bd691 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:43.724917 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9e41d-48d6-4c73-9055-92165a7bd691 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:43.733150 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c9e41d-48d6-4c73-9055-92165a7bd691 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:43.734633 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c9e41d-48d6-4c73-9055-92165a7bd691 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:43.746299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-87c9e41d-48d6-4c73-9055-92165a7bd691 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 22:35:43.746299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:43.746299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:43.746299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:43.746299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:43.746299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:43.746299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:43.746299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:43.746299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:43.746299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:43.746299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:43.746299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:43.746299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:35:43.746299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:35:43.746299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:35:43.746299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:35:43.747302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:43.747302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:43.747302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:35:43.747302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:43.747302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:35:43.747302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:43.747302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:43.747302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:43.747302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:43.747302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:43.747302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:43.747302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:43.747302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:43.747302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:43.747302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:43.747302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:43.747302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:43.747302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:43.748262 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:43.748262 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:43.748262 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:35:43.748262 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:35:43.748262 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:35:43.748262 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:35:43.748262 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:35:43.748262 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:35:43.748262 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 22:35:43.748262 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:43.748262 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 472/940] 10.222.0.76 () {56 vars in 1117 bytes} [Wed Aug 10 22:35:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:43.748262 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9e41d-48d6-4c73-9055-92165a7bd691 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:43.766759 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd170e1-0c29-4e75-8e73-0eb3433edca0 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:43.782940 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:43.806426 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f185d2-a4d6-4256-b52a-98ba288c7ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:43.806964 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f185d2-a4d6-4256-b52a-98ba288c7ae0 None admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:43.807244 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f185d2-a4d6-4256-b52a-98ba288c7ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:43.807585 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f185d2-a4d6-4256-b52a-98ba288c7ae0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:43.808603 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f185d2-a4d6-4256-b52a-98ba288c7ae0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:43.810258 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f185d2-a4d6-4256-b52a-98ba288c7ae0 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:43.845320 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 469/941] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:35:43 2022] POST /identity/v3/endpoints => generated 347 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:35:43.845320 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f185d2-a4d6-4256-b52a-98ba288c7ae0 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:43.882575 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9e41d-48d6-4c73-9055-92165a7bd691 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:43.903674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:43.939324 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c337c80-59df-4b21-b18c-d927c75a886b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:43.940882 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c337c80-59df-4b21-b18c-d927c75a886b None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:43.941559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c337c80-59df-4b21-b18c-d927c75a886b None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:43.941965 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c337c80-59df-4b21-b18c-d927c75a886b None admin] PATH_INFO: `/v3/projects/service` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:43.953521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c337c80-59df-4b21-b18c-d927c75a886b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:43.955256 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c337c80-59df-4b21-b18c-d927c75a886b None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:43.973070 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-3c337c80-59df-4b21-b18c-d927c75a886b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:43.973070 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:43.973070 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:43.973070 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:43.973070 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:43.973070 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:43.973070 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:43.973070 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:43.973070 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:43.973070 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:43.973070 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:43.973070 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:43.973070 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:35:43.973070 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:35:43.973070 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:35:43.974190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:35:43.974190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:43.974190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:43.974190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:43.974190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:43.974190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:43.974190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:43.974190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:43.974190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:43.974190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:43.974190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:43.974190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:43.974190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:43.974190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:43.974190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:43.974190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:43.974190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:43.975066 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:35:43.975066 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:35:43.975066 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:35:43.975066 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:35:43.975066 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:35:43.975066 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:35:43.975066 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:43.975066 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:43.975740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f185d2-a4d6-4256-b52a-98ba288c7ae0 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:43.975835 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 473/942] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:35:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:43.981998 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c337c80-59df-4b21-b18c-d927c75a886b None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:44.017980 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bab8cb-fb19-48d4-82ec-bf3da1d30e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:44.018708 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bab8cb-fb19-48d4-82ec-bf3da1d30e82 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:44.019085 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bab8cb-fb19-48d4-82ec-bf3da1d30e82 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:44.019498 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bab8cb-fb19-48d4-82ec-bf3da1d30e82 None admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:44.020498 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bab8cb-fb19-48d4-82ec-bf3da1d30e82 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:44.020872 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bab8cb-fb19-48d4-82ec-bf3da1d30e82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:44.022425 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bab8cb-fb19-48d4-82ec-bf3da1d30e82 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:44.043238 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 470/943] 10.222.0.76 () {56 vars in 1149 bytes} [Wed Aug 10 22:35:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:44.049513 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bab8cb-fb19-48d4-82ec-bf3da1d30e82 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:44.090463 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:44.120844 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c337c80-59df-4b21-b18c-d927c75a886b None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:44.160229 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665c9c0-bff7-4886-821d-75c60094d89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:44.161546 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665c9c0-bff7-4886-821d-75c60094d89d None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:44.162285 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665c9c0-bff7-4886-821d-75c60094d89d None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:44.162796 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665c9c0-bff7-4886-821d-75c60094d89d None admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:44.164371 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8665c9c0-bff7-4886-821d-75c60094d89d None admin] RBAC: Adding query filter params (name=service) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:44.164851 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8665c9c0-bff7-4886-821d-75c60094d89d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:44.168162 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8665c9c0-bff7-4886-821d-75c60094d89d None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:44.177228 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bab8cb-fb19-48d4-82ec-bf3da1d30e82 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:44.181094 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:44.197331 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 474/944] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 10 22:35:43 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:44.197331 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b47b3-1790-4ad7-9d0f-e296df2b0e0d None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:44.197331 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b47b3-1790-4ad7-9d0f-e296df2b0e0d None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:44.197331 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b47b3-1790-4ad7-9d0f-e296df2b0e0d None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:44.199010 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 475/945] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:44.208693 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e715fb-0da4-4234-aa87-1ea79aa493c0 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:44.209047 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e715fb-0da4-4234-aa87-1ea79aa493c0 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:44.209349 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e715fb-0da4-4234-aa87-1ea79aa493c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:44.231234 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6f71c-cc19-4ee3-8f48-2f56644b226d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:44.232391 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6f71c-cc19-4ee3-8f48-2f56644b226d None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:44.232850 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6f71c-cc19-4ee3-8f48-2f56644b226d None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:44.233385 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6f71c-cc19-4ee3-8f48-2f56644b226d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:44.234463 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-35e715fb-0da4-4234-aa87-1ea79aa493c0 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:44.235717 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-35e715fb-0da4-4234-aa87-1ea79aa493c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:44.247015 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c6f71c-cc19-4ee3-8f48-2f56644b226d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:44.248810 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c6f71c-cc19-4ee3-8f48-2f56644b226d None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:44.266253 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-41c6f71c-cc19-4ee3-8f48-2f56644b226d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:44.266253 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:44.266253 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:44.266253 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:44.266253 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:44.266253 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:44.266253 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:44.266253 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:44.266253 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:44.266253 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:44.266253 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:44.266253 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:44.266253 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:44.266253 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:44.266253 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:44.266253 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:44.267249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:44.267249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:44.267249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:44.267249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:44.267249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:44.267249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:44.267249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:44.267249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:44.267249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:44.267249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:44.267249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:44.267249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:44.267249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:44.267249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:44.267249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:44.267249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:44.267249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:44.268170 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:44.268170 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:44.268170 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:44.270510 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 471/946] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:44.276579 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6f71c-cc19-4ee3-8f48-2f56644b226d None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:44.336452 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 476/947] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:35:44.360123 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e715fb-0da4-4234-aa87-1ea79aa493c0 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:44.407502 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6f71c-cc19-4ee3-8f48-2f56644b226d None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:44.447155 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:44.453904 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a137e-29bd-4041-9bcd-cf991a4c414c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:44.454523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a137e-29bd-4041-9bcd-cf991a4c414c None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:44.454801 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a137e-29bd-4041-9bcd-cf991a4c414c None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:44.455073 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a137e-29bd-4041-9bcd-cf991a4c414c None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:44.455719 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3a137e-29bd-4041-9bcd-cf991a4c414c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:44.456016 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3a137e-29bd-4041-9bcd-cf991a4c414c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:44.458275 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3a137e-29bd-4041-9bcd-cf991a4c414c None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:44.475446 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 472/948] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:35:44.481719 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a137e-29bd-4041-9bcd-cf991a4c414c None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:44.489864 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e715fb-0da4-4234-aa87-1ea79aa493c0 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:44.506148 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:44.536263 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf59f1e-c46e-41b2-87be-0d73cb82fa86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:44.536906 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf59f1e-c46e-41b2-87be-0d73cb82fa86 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:44.537218 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf59f1e-c46e-41b2-87be-0d73cb82fa86 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:44.537961 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf59f1e-c46e-41b2-87be-0d73cb82fa86 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:44.538739 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf59f1e-c46e-41b2-87be-0d73cb82fa86 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:44.540579 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf59f1e-c46e-41b2-87be-0d73cb82fa86 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:44.552079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-bdf59f1e-c46e-41b2-87be-0d73cb82fa86 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 22:35:44.552079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:44.552079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:35:44.552079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:35:44.552079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:35:44.552079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:35:44.552079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:35:44.552079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:44.552079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 22:35:44.552079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:44.552079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:35:44.552079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:44.552079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:44.552079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:44.552079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:44.552079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:44.552079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:44.553139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:44.553139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:44.553139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:44.553139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:44.553139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:44.553139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:44.553139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:35:44.553139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:35:44.553139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:35:44.553139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:35:44.553139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:44.553139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:44.553139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:44.553139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:44.553139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:44.553139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:44.559847 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:44.559847 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:44.559847 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:44.559847 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:44.559847 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:44.559847 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:44.559847 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:44.559847 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:44.559847 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:44.559847 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:44.559847 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:35:44.559847 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:44.559847 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 22:35:44.559847 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:44.559847 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 477/949] 10.222.0.76 () {56 vars in 1126 bytes} [Wed Aug 10 22:35:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:35:44.562736 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf59f1e-c46e-41b2-87be-0d73cb82fa86 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:44.608783 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a137e-29bd-4041-9bcd-cf991a4c414c None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:44.646279 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d691cd-a949-4c9a-8d9c-c83125d98eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:44.646830 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d691cd-a949-4c9a-8d9c-c83125d98eea None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:44.647157 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d691cd-a949-4c9a-8d9c-c83125d98eea None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:44.647430 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d691cd-a949-4c9a-8d9c-c83125d98eea None admin] PATH_INFO: `/v3/projects/service` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:44.656337 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d691cd-a949-4c9a-8d9c-c83125d98eea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:44.657892 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d691cd-a949-4c9a-8d9c-c83125d98eea None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:44.670238 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-a0d691cd-a949-4c9a-8d9c-c83125d98eea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:44.670238 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:44.670238 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:44.670238 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:44.670238 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:44.670238 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:44.670238 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:44.670238 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:44.670238 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:44.670238 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:44.670238 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:44.670238 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:44.670238 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:35:44.670238 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:35:44.670238 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:35:44.671201 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:35:44.671201 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:44.671201 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:44.671201 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:44.671201 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:44.671201 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:44.671201 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:44.671201 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:44.671201 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:44.671201 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:44.671201 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:44.671201 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:44.671201 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:44.671201 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:44.671201 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:44.671201 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:44.671201 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:44.672124 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:35:44.672124 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:35:44.672124 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:35:44.672124 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:35:44.672124 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:35:44.672124 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:35:44.672124 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:44.672124 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:44.676191 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 473/950] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:35:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:35:44.677178 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d691cd-a949-4c9a-8d9c-c83125d98eea None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:44.699144 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf59f1e-c46e-41b2-87be-0d73cb82fa86 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:44.739226 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:44.744081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c7a6a7-1b07-45f3-b57d-378995872937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:44.745610 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c7a6a7-1b07-45f3-b57d-378995872937 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:44.746191 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c7a6a7-1b07-45f3-b57d-378995872937 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:44.746663 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c7a6a7-1b07-45f3-b57d-378995872937 None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:44.747631 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c7a6a7-1b07-45f3-b57d-378995872937 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:44.748610 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c7a6a7-1b07-45f3-b57d-378995872937 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:44.750516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c7a6a7-1b07-45f3-b57d-378995872937 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:44.762435 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 478/951] 10.222.0.76 () {56 vars in 1122 bytes} [Wed Aug 10 22:35:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:44.764947 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:44.765303 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c7a6a7-1b07-45f3-b57d-378995872937 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:44.792224 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d691cd-a949-4c9a-8d9c-c83125d98eea None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:44.831854 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5f6a67-7ad8-40bc-8d94-e7f71ce4059e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:44.832466 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f6a67-7ad8-40bc-8d94-e7f71ce4059e None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:44.832908 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f6a67-7ad8-40bc-8d94-e7f71ce4059e None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:44.833505 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f6a67-7ad8-40bc-8d94-e7f71ce4059e None admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:44.834294 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5f6a67-7ad8-40bc-8d94-e7f71ce4059e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:44.834723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5f6a67-7ad8-40bc-8d94-e7f71ce4059e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:44.836419 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5f6a67-7ad8-40bc-8d94-e7f71ce4059e None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:44.857191 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 474/952] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:35:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:44.859031 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5f6a67-7ad8-40bc-8d94-e7f71ce4059e None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:44.917642 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c7a6a7-1b07-45f3-b57d-378995872937 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:44.964178 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16429db7-c589-4d41-b986-ff16a0f337b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:44.964944 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16429db7-c589-4d41-b986-ff16a0f337b2 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:44.965347 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16429db7-c589-4d41-b986-ff16a0f337b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:44.965734 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16429db7-c589-4d41-b986-ff16a0f337b2 None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:44.966471 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16429db7-c589-4d41-b986-ff16a0f337b2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:44.966876 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16429db7-c589-4d41-b986-ff16a0f337b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:44.968564 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16429db7-c589-4d41-b986-ff16a0f337b2 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:44.977467 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 479/953] 10.222.0.76 () {56 vars in 1122 bytes} [Wed Aug 10 22:35:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:44.981462 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16429db7-c589-4d41-b986-ff16a0f337b2 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:44.991619 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5f6a67-7ad8-40bc-8d94-e7f71ce4059e None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:45.030154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5006e911-bac7-471c-9cae-0f67a7ef1128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:45.030403 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5006e911-bac7-471c-9cae-0f67a7ef1128 None admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:45.030630 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5006e911-bac7-471c-9cae-0f67a7ef1128 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:45.030870 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5006e911-bac7-471c-9cae-0f67a7ef1128 None admin] PATH_INFO: `/v3/projects/3caa2ea846f7449b8c483788fc5846c8/users/aff5e804c64345b298c745287a1d0a32/roles/ee973ce49dc142a6b0763794e078333d` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:45.033115 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5006e911-bac7-471c-9cae-0f67a7ef1128 None admin] RBAC: Authorizing `identity:create_grant(project_id=3caa2ea846f7449b8c483788fc5846c8, user_id=aff5e804c64345b298c745287a1d0a32, role_id=ee973ce49dc142a6b0763794e078333d)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:45.034665 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5006e911-bac7-471c-9cae-0f67a7ef1128 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:45.038220 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:45.061306 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 475/954] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 10 22:35:44 2022] PUT /identity/v3/projects/3caa2ea846f7449b8c483788fc5846c8/users/aff5e804c64345b298c745287a1d0a32/roles/ee973ce49dc142a6b0763794e078333d => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:35:45.078598 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:45.131553 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16429db7-c589-4d41-b986-ff16a0f337b2 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:45.170939 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1539c23-fdc9-43f6-a99c-63f971f5ba2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:45.171966 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1539c23-fdc9-43f6-a99c-63f971f5ba2b None admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:45.172424 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1539c23-fdc9-43f6-a99c-63f971f5ba2b None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:45.172873 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1539c23-fdc9-43f6-a99c-63f971f5ba2b None admin] PATH_INFO: `/v3/endpoints` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:45.173743 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1539c23-fdc9-43f6-a99c-63f971f5ba2b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:45.175319 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1539c23-fdc9-43f6-a99c-63f971f5ba2b None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:45.205455 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 480/955] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:35:44 2022] POST /identity/v3/endpoints => generated 349 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:35:46.033474 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29f105-d83b-4f5e-99cd-a7187bbe8269 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:46.033996 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29f105-d83b-4f5e-99cd-a7187bbe8269 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:46.033996 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29f105-d83b-4f5e-99cd-a7187bbe8269 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:46.035488 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 476/956] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:46.042646 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af1a49-9e68-4d78-b8d8-c5796828f970 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:46.043244 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af1a49-9e68-4d78-b8d8-c5796828f970 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:46.043729 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af1a49-9e68-4d78-b8d8-c5796828f970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:46.067931 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-a4af1a49-9e68-4d78-b8d8-c5796828f970 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:46.069573 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-a4af1a49-9e68-4d78-b8d8-c5796828f970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:46.162295 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 481/957] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:35:46.188873 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29f105-d83b-4f5e-99cd-a7187bbe8269 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:46.320919 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29f105-d83b-4f5e-99cd-a7187bbe8269 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:46.362617 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba54ee4a-682f-4910-9148-d1272e430106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:46.363317 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54ee4a-682f-4910-9148-d1272e430106 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:46.363788 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54ee4a-682f-4910-9148-d1272e430106 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:46.365523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54ee4a-682f-4910-9148-d1272e430106 None admin] PATH_INFO: `/v3/services/compute` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:46.365523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba54ee4a-682f-4910-9148-d1272e430106 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:46.366385 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba54ee4a-682f-4910-9148-d1272e430106 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:46.377038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-ba54ee4a-682f-4910-9148-d1272e430106 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 22:35:46.377038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:46.377038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:35:46.377038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:35:46.377038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:35:46.377038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:35:46.377038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:35:46.377038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:46.377038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 22:35:46.377038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:46.377038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:35:46.377038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:46.377038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:46.377038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:46.377038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:46.377038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:46.377038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:46.378144 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:46.378144 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:46.378144 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:46.378144 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:46.378144 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:46.378144 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:46.378144 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:35:46.378144 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:35:46.378144 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:35:46.378144 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:35:46.378144 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:46.378144 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:46.378144 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:46.378144 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:46.378144 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:46.378144 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:46.378996 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:46.378996 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:46.378996 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:46.378996 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:46.378996 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:46.378996 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:46.378996 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:46.378996 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:46.378996 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:46.378996 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:46.378996 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:35:46.378996 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:46.378996 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 22:35:46.378996 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:46.378996 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 477/958] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:35:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:35:46.387227 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af1a49-9e68-4d78-b8d8-c5796828f970 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:46.464584 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:46.536240 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af1a49-9e68-4d78-b8d8-c5796828f970 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:46.549963 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c8692-0723-43c5-9dd8-2455215973a4 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:46.550198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c8692-0723-43c5-9dd8-2455215973a4 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:46.550443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c8692-0723-43c5-9dd8-2455215973a4 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:46.552093 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 478/959] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:35:46.558848 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b58e94-c47e-4f34-9172-e4f9085396fc None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:46.559125 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b58e94-c47e-4f34-9172-e4f9085396fc None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:46.559490 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b58e94-c47e-4f34-9172-e4f9085396fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:46.586537 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-91b58e94-c47e-4f34-9172-e4f9085396fc None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:46.589122 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d937b-c29b-4ca3-9b31-76ffd95667ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:46.589706 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-91b58e94-c47e-4f34-9172-e4f9085396fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:46.590167 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d937b-c29b-4ca3-9b31-76ffd95667ca None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:46.590618 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d937b-c29b-4ca3-9b31-76ffd95667ca None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:46.590984 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d937b-c29b-4ca3-9b31-76ffd95667ca None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:46.591842 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d937b-c29b-4ca3-9b31-76ffd95667ca None admin] RBAC: Adding query filter params (name=compute) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:46.592143 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d937b-c29b-4ca3-9b31-76ffd95667ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:46.593555 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d937b-c29b-4ca3-9b31-76ffd95667ca None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:46.599423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 482/960] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 10 22:35:46 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:46.608440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d937b-c29b-4ca3-9b31-76ffd95667ca None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:46.665463 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:46.678844 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 479/961] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:35:46.710268 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b58e94-c47e-4f34-9172-e4f9085396fc None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:46.748497 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d937b-c29b-4ca3-9b31-76ffd95667ca None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:46.762070 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:46.797063 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb080f9-dcee-4a45-8520-d1e312ab284e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:46.798102 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb080f9-dcee-4a45-8520-d1e312ab284e None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:46.798593 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb080f9-dcee-4a45-8520-d1e312ab284e None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:46.799150 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb080f9-dcee-4a45-8520-d1e312ab284e None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:46.800054 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb080f9-dcee-4a45-8520-d1e312ab284e None admin] RBAC: Adding query filter params (type=compute) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:46.801586 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb080f9-dcee-4a45-8520-d1e312ab284e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:46.803812 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb080f9-dcee-4a45-8520-d1e312ab284e None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:46.813136 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 483/962] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 10 22:35:46 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:46.834820 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b58e94-c47e-4f34-9172-e4f9085396fc None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:46.877765 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3d595-0852-4e5a-96ae-816724695d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:46.878363 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3d595-0852-4e5a-96ae-816724695d23 None admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:46.878637 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3d595-0852-4e5a-96ae-816724695d23 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:46.878905 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3d595-0852-4e5a-96ae-816724695d23 None admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:46.879587 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3d595-0852-4e5a-96ae-816724695d23 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:46.881045 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3d595-0852-4e5a-96ae-816724695d23 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:46.899407 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.sql.core [None req-e0e3d595-0852-4e5a-96ae-816724695d23 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 22:35:46.899407 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 10 22:35:46.899407 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [parameters: {'user_id': '6cfb0ed3df9c49eca5f814ddcc13f222', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 10 22:35:46.899407 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93439) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 22:35:46.906792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-e0e3d595-0852-4e5a-96ae-816724695d23 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 10 22:35:46.906792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:46.906792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 22:35:46.906792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 22:35:46.906792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 22:35:46.906792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 22:35:46.906792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 22:35:46.906792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 22:35:46.906792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 22:35:46.906792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application conn.query(q) Aug 10 22:35:46.906792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 22:35:46.906792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 22:35:46.906792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 22:35:46.906792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application result.read() Aug 10 22:35:46.906792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 22:35:46.908515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 22:35:46.908515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 22:35:46.908515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 22:35:46.908515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 22:35:46.908515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 22:35:46.908515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 22:35:46.908515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 22:35:46.908515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 22:35:46.908515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:46.908515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 22:35:46.908515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:46.908515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:46.908515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 22:35:46.908515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 22:35:46.908515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 10 22:35:46.908515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 10 22:35:46.908515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 22:35:46.908515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application next(self.gen) Aug 10 22:35:46.910515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 22:35:46.910515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application yield resource Aug 10 22:35:46.910515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 22:35:46.910515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application next(self.gen) Aug 10 22:35:46.910515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 22:35:46.910515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 22:35:46.910515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 22:35:46.910515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 22:35:46.910515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 22:35:46.910515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise self.value Aug 10 22:35:46.910515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 22:35:46.910515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 22:35:46.910515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 22:35:46.910515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application session.commit() Aug 10 22:35:46.910515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 22:35:46.910515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 22:35:46.910515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 22:35:46.912362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 22:35:46.912362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 22:35:46.912362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self.session.flush() Aug 10 22:35:46.912362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 22:35:46.912362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 22:35:46.912362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 22:35:46.912362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 22:35:46.912362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 22:35:46.912362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application compat.raise_( Aug 10 22:35:46.912362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 22:35:46.912362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception Aug 10 22:35:46.912362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 22:35:46.912362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 22:35:46.912362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 22:35:46.912362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 22:35:46.912362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 22:35:46.912362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 22:35:46.916652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 22:35:46.916652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 22:35:46.916652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 10 22:35:46.916652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 10 22:35:46.916652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 22:35:46.916652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 22:35:46.916652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 22:35:46.916652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 22:35:46.916652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 22:35:46.916652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 22:35:46.916652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 22:35:46.916652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 22:35:46.916652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 22:35:46.916652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 22:35:46.916652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 22:35:46.916652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception Aug 10 22:35:46.922564 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 22:35:46.922564 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 22:35:46.922564 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 22:35:46.922564 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 22:35:46.922564 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 22:35:46.922564 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 22:35:46.922564 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 22:35:46.922564 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application conn.query(q) Aug 10 22:35:46.922564 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 22:35:46.922564 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 22:35:46.922564 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 22:35:46.922564 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application result.read() Aug 10 22:35:46.922564 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 22:35:46.922564 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 22:35:46.922564 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 22:35:46.922564 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 22:35:46.924600 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 22:35:46.924600 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 22:35:46.924600 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 22:35:46.924600 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 22:35:46.924600 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 22:35:46.924600 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 10 22:35:46.924600 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [parameters: {'user_id': '6cfb0ed3df9c49eca5f814ddcc13f222', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 10 22:35:46.924600 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 22:35:46.924600 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:46.924600 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:35:46.924600 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:46.924600 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:46.924600 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:46.924600 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:46.924600 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:46.932028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:46.932028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:46.932028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:46.932028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:46.932028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:46.932028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:46.932028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:46.932028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 10 22:35:46.932028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 10 22:35:46.932028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:46.932028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:46.932028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:35:46.932028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:46.932028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:35:46.932028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:46.932028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 10 22:35:46.932028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 10 22:35:46.937617 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 10 22:35:46.937617 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 10 22:35:46.937617 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 22:35:46.937617 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 22:35:46.937617 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 10 22:35:46.937617 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:46.937617 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 480/963] 10.222.0.76 () {60 vars in 1131 bytes} [Wed Aug 10 22:35:46 2022] POST /identity/v3/users => generated 160 bytes in 199 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:46.938578 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb080f9-dcee-4a45-8520-d1e312ab284e None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:47.066146 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb080f9-dcee-4a45-8520-d1e312ab284e None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:47.104380 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1e1db-bb5c-4824-a325-8a7662ce0e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:47.105031 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1e1db-bb5c-4824-a325-8a7662ce0e20 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:47.105359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1e1db-bb5c-4824-a325-8a7662ce0e20 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:47.105647 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1e1db-bb5c-4824-a325-8a7662ce0e20 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:47.113904 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c1e1db-bb5c-4824-a325-8a7662ce0e20 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:47.115418 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c1e1db-bb5c-4824-a325-8a7662ce0e20 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:47.128095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-52c1e1db-bb5c-4824-a325-8a7662ce0e20 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 22:35:47.128095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:47.128095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:47.128095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:47.128095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:47.128095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:47.128095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:47.128095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:47.128095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:47.128095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:47.128095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:47.128095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:47.128095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:35:47.128095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:35:47.128095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:35:47.128095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:35:47.129521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:47.129521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:47.129521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:35:47.129521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:47.129521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:35:47.129521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:47.129521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:47.129521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:47.129521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:47.129521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:47.129521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:47.129521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:47.129521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:47.129521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:47.129521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:47.129521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:47.129521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:47.129521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:47.132199 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:47.132199 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:47.132199 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:35:47.132199 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:35:47.132199 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:35:47.132199 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:35:47.132199 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:35:47.132199 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:35:47.132199 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 22:35:47.132199 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:47.132199 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 484/964] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:35:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:47.144772 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3d595-0852-4e5a-96ae-816724695d23 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:47.221258 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:47.279843 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3d595-0852-4e5a-96ae-816724695d23 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:47.296724 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a8eaa-cd48-4aa8-a730-e378f81c404c None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:47.297234 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a8eaa-cd48-4aa8-a730-e378f81c404c None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:47.297357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a8eaa-cd48-4aa8-a730-e378f81c404c None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:47.299020 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 485/965] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:47.304111 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e96582-7769-42f2-8313-3ae0cad672f0 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:47.304397 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e96582-7769-42f2-8313-3ae0cad672f0 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:47.304679 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e96582-7769-42f2-8313-3ae0cad672f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:47.319951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a97620e-f568-40a0-86fe-c7ea658581a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:47.320961 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97620e-f568-40a0-86fe-c7ea658581a3 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:47.321501 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97620e-f568-40a0-86fe-c7ea658581a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:47.322602 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97620e-f568-40a0-86fe-c7ea658581a3 None admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:47.323520 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a97620e-f568-40a0-86fe-c7ea658581a3 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:47.324026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a97620e-f568-40a0-86fe-c7ea658581a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:47.325925 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a97620e-f568-40a0-86fe-c7ea658581a3 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:47.326289 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-91e96582-7769-42f2-8313-3ae0cad672f0 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:47.327507 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-91e96582-7769-42f2-8313-3ae0cad672f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:47.356092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:47.367866 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 481/966] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 10 22:35:47 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:47.412466 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 486/967] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:35:47.438121 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a97620e-f568-40a0-86fe-c7ea658581a3 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:47.529871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210a355-ff5f-48c6-9ddc-fe8c3e713fe9 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:47.530490 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210a355-ff5f-48c6-9ddc-fe8c3e713fe9 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:47.530971 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210a355-ff5f-48c6-9ddc-fe8c3e713fe9 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:47.532774 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 487/968] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:47.538263 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7880b4-3618-4e09-9872-d531305da1a8 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:47.538557 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7880b4-3618-4e09-9872-d531305da1a8 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:47.538835 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7880b4-3618-4e09-9872-d531305da1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:47.563458 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a97620e-f568-40a0-86fe-c7ea658581a3 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:47.564081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-db7880b4-3618-4e09-9872-d531305da1a8 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:47.565792 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-db7880b4-3618-4e09-9872-d531305da1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:47.576266 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:47.611788 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cb508f-8afe-42f3-8a2a-29104fc09429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:47.612444 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb508f-8afe-42f3-8a2a-29104fc09429 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:47.612731 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb508f-8afe-42f3-8a2a-29104fc09429 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:47.613007 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb508f-8afe-42f3-8a2a-29104fc09429 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:47.631235 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cb508f-8afe-42f3-8a2a-29104fc09429 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:35:47.637777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-60cb508f-8afe-42f3-8a2a-29104fc09429 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 22:35:47.637777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:47.637777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:47.637777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:47.637777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:47.637777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:47.637777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:47.637777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:47.637777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:47.637777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:47.637777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:47.637777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:47.637777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:35:47.637777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:35:47.637777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:35:47.637777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:35:47.638771 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:35:47.638771 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:35:47.638771 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:35:47.638771 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:35:47.638771 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:47.638771 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:47.638771 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:47.638771 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:47.638771 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:47.638771 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:47.638771 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:47.638771 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:47.638771 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:47.638771 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:47.638771 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:47.638771 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:47.639804 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:47.639804 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:47.639804 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:47.639804 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:47.639804 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:35:47.639804 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:35:47.639804 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:35:47.639804 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:35:47.639804 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:35:47.639804 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:35:47.639804 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 22:35:47.639804 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:47.639804 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 482/969] 10.222.0.76 () {56 vars in 1120 bytes} [Wed Aug 10 22:35:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:35:47.645204 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cb508f-8afe-42f3-8a2a-29104fc09429 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:47.654096 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 488/970] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:35:47.676484 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7880b4-3618-4e09-9872-d531305da1a8 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:47.753659 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cb508f-8afe-42f3-8a2a-29104fc09429 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:47.775486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:47.786306 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7880b4-3618-4e09-9872-d531305da1a8 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:47.791581 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd8c97e-560c-4264-bcf8-8c6f6959a5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:47.792192 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd8c97e-560c-4264-bcf8-8c6f6959a5e9 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:47.792467 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd8c97e-560c-4264-bcf8-8c6f6959a5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:47.792736 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd8c97e-560c-4264-bcf8-8c6f6959a5e9 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:47.793420 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd8c97e-560c-4264-bcf8-8c6f6959a5e9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:47.793720 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd8c97e-560c-4264-bcf8-8c6f6959a5e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:47.795079 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd8c97e-560c-4264-bcf8-8c6f6959a5e9 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:47.804170 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 483/971] 10.222.0.76 () {56 vars in 1115 bytes} [Wed Aug 10 22:35:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:35:47.809984 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd8c97e-560c-4264-bcf8-8c6f6959a5e9 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:47.830097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec5d2e-7ef3-47f1-8a18-6d83678e1d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:47.830857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec5d2e-7ef3-47f1-8a18-6d83678e1d7f None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:47.831141 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec5d2e-7ef3-47f1-8a18-6d83678e1d7f None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:47.831524 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec5d2e-7ef3-47f1-8a18-6d83678e1d7f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:47.832109 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ec5d2e-7ef3-47f1-8a18-6d83678e1d7f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:47.833507 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ec5d2e-7ef3-47f1-8a18-6d83678e1d7f None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:47.843285 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-21ec5d2e-7ef3-47f1-8a18-6d83678e1d7f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 22:35:47.843285 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:47.843285 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:35:47.843285 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:35:47.843285 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:35:47.843285 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:35:47.843285 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:35:47.843285 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:47.843285 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 22:35:47.843285 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:47.843285 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:35:47.843285 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:47.843285 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:47.843285 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:47.843285 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:47.843285 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:47.843285 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:47.844297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:47.844297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:47.844297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:47.844297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:47.844297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:47.844297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:47.844297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:35:47.844297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:35:47.844297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:35:47.844297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:35:47.844297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:47.844297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:47.844297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:47.844297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:47.844297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:47.844297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:47.845138 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:47.845138 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:47.845138 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:47.845138 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:47.845138 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:47.845138 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:47.845138 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:47.845138 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:47.845138 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:47.845138 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:47.845138 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:35:47.845138 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:47.845138 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 22:35:47.845138 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:47.845138 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 489/972] 10.222.0.76 () {56 vars in 1126 bytes} [Wed Aug 10 22:35:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:35:47.850942 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec5d2e-7ef3-47f1-8a18-6d83678e1d7f None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:47.851102 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:47.921846 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd8c97e-560c-4264-bcf8-8c6f6959a5e9 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:47.958318 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c97552-7047-4e36-8276-0dce2acfa39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:47.958870 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c97552-7047-4e36-8276-0dce2acfa39b None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:47.959212 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c97552-7047-4e36-8276-0dce2acfa39b None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:47.959411 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c97552-7047-4e36-8276-0dce2acfa39b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:47.967559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec5d2e-7ef3-47f1-8a18-6d83678e1d7f None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:47.970500 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c97552-7047-4e36-8276-0dce2acfa39b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:47.971840 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c97552-7047-4e36-8276-0dce2acfa39b None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:47.986556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-41c97552-7047-4e36-8276-0dce2acfa39b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:47.986556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:47.986556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:47.986556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:47.986556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:47.986556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:47.986556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:47.986556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:47.986556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:47.986556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:47.986556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:47.986556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:47.986556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:47.986556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:47.986556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:47.986556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:47.987638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:47.987638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:47.987638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:47.987638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:47.987638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:47.987638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:47.987638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:47.987638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:47.987638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:47.987638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:47.987638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:47.987638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:47.987638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:47.987638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:47.987638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:47.987638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:47.987638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:47.989349 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:47.989349 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:47.989349 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:47.989349 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 484/973] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:47.994793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c97552-7047-4e36-8276-0dce2acfa39b None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:48.004686 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d88e4-6d9d-4c78-ade3-b467ea26a5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:48.005325 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d88e4-6d9d-4c78-ade3-b467ea26a5ac None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:48.005606 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d88e4-6d9d-4c78-ade3-b467ea26a5ac None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:48.005872 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d88e4-6d9d-4c78-ade3-b467ea26a5ac None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:48.006502 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8d88e4-6d9d-4c78-ade3-b467ea26a5ac None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:48.006794 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8d88e4-6d9d-4c78-ade3-b467ea26a5ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:48.008168 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8d88e4-6d9d-4c78-ade3-b467ea26a5ac None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:48.013848 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 490/974] 10.222.0.76 () {56 vars in 1122 bytes} [Wed Aug 10 22:35:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:35:48.019174 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d88e4-6d9d-4c78-ade3-b467ea26a5ac None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:48.041848 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:48.093749 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:48.113039 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c97552-7047-4e36-8276-0dce2acfa39b None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:48.131309 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d88e4-6d9d-4c78-ade3-b467ea26a5ac None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:48.150404 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b04f0-aad3-4739-92dc-e0cc314c7d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:48.151164 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b04f0-aad3-4739-92dc-e0cc314c7d0b None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:48.151420 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b04f0-aad3-4739-92dc-e0cc314c7d0b None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:48.151686 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b04f0-aad3-4739-92dc-e0cc314c7d0b None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:48.152488 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91b04f0-aad3-4739-92dc-e0cc314c7d0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:48.152597 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91b04f0-aad3-4739-92dc-e0cc314c7d0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:48.153971 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91b04f0-aad3-4739-92dc-e0cc314c7d0b None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:48.167185 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 485/975] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:48.170246 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9973d-e4c1-42ee-9a6f-321f578df8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:48.170859 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9973d-e4c1-42ee-9a6f-321f578df8d9 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:48.171133 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9973d-e4c1-42ee-9a6f-321f578df8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:48.171400 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9973d-e4c1-42ee-9a6f-321f578df8d9 None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:48.172024 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd9973d-e4c1-42ee-9a6f-321f578df8d9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:48.172322 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd9973d-e4c1-42ee-9a6f-321f578df8d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:48.173120 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b04f0-aad3-4739-92dc-e0cc314c7d0b None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:48.173711 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd9973d-e4c1-42ee-9a6f-321f578df8d9 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:48.178803 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 491/976] 10.222.0.76 () {56 vars in 1122 bytes} [Wed Aug 10 22:35:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:35:48.184444 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9973d-e4c1-42ee-9a6f-321f578df8d9 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:48.273441 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:48.283384 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b04f0-aad3-4739-92dc-e0cc314c7d0b None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:48.293205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9973d-e4c1-42ee-9a6f-321f578df8d9 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:48.320707 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd799fe3-c4cd-46af-ab00-541921f3e9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:48.322004 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd799fe3-c4cd-46af-ab00-541921f3e9c6 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:48.322292 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd799fe3-c4cd-46af-ab00-541921f3e9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:48.322556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd799fe3-c4cd-46af-ab00-541921f3e9c6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:48.329948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd799fe3-c4cd-46af-ab00-541921f3e9c6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:48.331386 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd799fe3-c4cd-46af-ab00-541921f3e9c6 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:48.332273 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216868c0-c431-431d-b63f-44d5437edebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:48.333052 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216868c0-c431-431d-b63f-44d5437edebf None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:48.333250 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216868c0-c431-431d-b63f-44d5437edebf None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:48.333529 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216868c0-c431-431d-b63f-44d5437edebf None admin] PATH_INFO: `/v3/endpoints` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:48.334241 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216868c0-c431-431d-b63f-44d5437edebf None admin] RBAC: Adding query filter params (interface=admin, service_id=a74afca16248445e951e92a5dccfa41b, region_id=RegionOne) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:48.334841 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216868c0-c431-431d-b63f-44d5437edebf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:48.335900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216868c0-c431-431d-b63f-44d5437edebf None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:48.347158 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 492/977] 10.222.0.76 () {56 vars in 1249 bytes} [Wed Aug 10 22:35:48 2022] GET /identity/v3/endpoints?service_id=a74afca16248445e951e92a5dccfa41b&interface=admin®ion_id=RegionOne => generated 193 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:35:48.347317 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-fd799fe3-c4cd-46af-ab00-541921f3e9c6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 22:35:48.347317 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:48.347317 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:48.347317 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:48.347317 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:48.347317 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:48.347317 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:48.347317 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:48.347317 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:48.347317 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:48.347317 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:48.347317 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:48.347317 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:35:48.347317 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:35:48.347317 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:35:48.347317 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:35:48.348190 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:48.348190 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:48.348190 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:35:48.348190 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:48.348190 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:35:48.348190 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:48.348190 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:48.348190 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:48.348190 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:48.348190 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:48.348190 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:48.348190 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:48.348190 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:48.348190 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:48.348190 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:48.348190 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:48.348190 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:48.348190 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:48.349154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:48.349154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:48.349154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:35:48.349154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:35:48.349154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:35:48.349154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:35:48.349154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:35:48.349154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:35:48.349154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 22:35:48.349154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:48.349763 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216868c0-c431-431d-b63f-44d5437edebf None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:48.350464 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 486/978] 10.222.0.76 () {56 vars in 1117 bytes} [Wed Aug 10 22:35:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:35:48.351248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd799fe3-c4cd-46af-ab00-541921f3e9c6 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:48.355600 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:48.463356 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216868c0-c431-431d-b63f-44d5437edebf None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:48.479668 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd799fe3-c4cd-46af-ab00-541921f3e9c6 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:48.502277 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec391e-ddc6-4359-8c78-8a3d59205f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:48.502983 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec391e-ddc6-4359-8c78-8a3d59205f3a None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:48.503187 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec391e-ddc6-4359-8c78-8a3d59205f3a None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:48.503462 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec391e-ddc6-4359-8c78-8a3d59205f3a None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:48.504073 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ec391e-ddc6-4359-8c78-8a3d59205f3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:48.505484 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ec391e-ddc6-4359-8c78-8a3d59205f3a None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:48.521316 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 493/979] 10.222.0.76 () {56 vars in 1087 bytes} [Wed Aug 10 22:35:48 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 22:35:48.521548 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2df2d9-b2a2-4215-8a91-862550c1b71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:48.521548 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2df2d9-b2a2-4215-8a91-862550c1b71d None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:48.521548 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2df2d9-b2a2-4215-8a91-862550c1b71d None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:48.521548 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2df2d9-b2a2-4215-8a91-862550c1b71d None admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:48.521892 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2df2d9-b2a2-4215-8a91-862550c1b71d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:48.521892 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2df2d9-b2a2-4215-8a91-862550c1b71d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:48.521892 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2df2d9-b2a2-4215-8a91-862550c1b71d None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:48.541694 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 487/980] 10.222.0.76 () {56 vars in 1149 bytes} [Wed Aug 10 22:35:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:48.547470 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec391e-ddc6-4359-8c78-8a3d59205f3a None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:48.560974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:48.659380 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec391e-ddc6-4359-8c78-8a3d59205f3a None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:48.696886 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242cdaa2-9cd1-4146-925d-0660a0e989bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:48.697614 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242cdaa2-9cd1-4146-925d-0660a0e989bd None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:48.697946 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242cdaa2-9cd1-4146-925d-0660a0e989bd None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:48.698223 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242cdaa2-9cd1-4146-925d-0660a0e989bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:48.768931 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242cdaa2-9cd1-4146-925d-0660a0e989bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:48.770429 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242cdaa2-9cd1-4146-925d-0660a0e989bd None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:48.782647 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:48.873566 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 494/981] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:35:48 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 22:35:48.879074 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2df2d9-b2a2-4215-8a91-862550c1b71d None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:48.944645 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:48.947296 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faedd383-d34d-4f19-9db3-c548142a65d1 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:48.947708 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faedd383-d34d-4f19-9db3-c548142a65d1 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:48.948024 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faedd383-d34d-4f19-9db3-c548142a65d1 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:48.949508 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 495/982] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:35:48.956376 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b5ae9-a5a1-4287-833c-c5302e758a4f None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:48.956658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b5ae9-a5a1-4287-833c-c5302e758a4f None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:48.956931 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b5ae9-a5a1-4287-833c-c5302e758a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:48.981418 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-0c2b5ae9-a5a1-4287-833c-c5302e758a4f None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:48.983644 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-0c2b5ae9-a5a1-4287-833c-c5302e758a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:48.996729 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2df2d9-b2a2-4215-8a91-862550c1b71d None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:49.034281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133e6dd-760d-4ff1-96d7-7a9da557d043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:49.034917 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133e6dd-760d-4ff1-96d7-7a9da557d043 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:49.035195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133e6dd-760d-4ff1-96d7-7a9da557d043 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:49.035706 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133e6dd-760d-4ff1-96d7-7a9da557d043 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:49.046372 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9133e6dd-760d-4ff1-96d7-7a9da557d043 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:49.047824 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9133e6dd-760d-4ff1-96d7-7a9da557d043 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:49.061355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 496/983] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:35:49.062975 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-9133e6dd-760d-4ff1-96d7-7a9da557d043 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:49.062975 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:49.062975 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:49.062975 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:49.062975 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:49.062975 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:49.062975 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:49.062975 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:49.062975 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:49.062975 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:49.062975 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:49.062975 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:49.062975 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:49.062975 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:49.062975 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:49.062975 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:49.063934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:49.063934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:49.063934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:49.063934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:49.063934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:49.063934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:49.063934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:49.063934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:49.063934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:49.063934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:49.063934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:49.063934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:49.063934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:49.063934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:49.063934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:49.063934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:49.063934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:49.065127 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:49.065127 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:49.065127 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:49.065127 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 488/984] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:49.069887 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b5ae9-a5a1-4287-833c-c5302e758a4f None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:49.084593 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133e6dd-760d-4ff1-96d7-7a9da557d043 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:49.118892 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:49.187696 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b5ae9-a5a1-4287-833c-c5302e758a4f None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:49.192727 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133e6dd-760d-4ff1-96d7-7a9da557d043 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:49.207859 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:49.225399 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae52633-0e3d-4857-85e3-efdce55483d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:49.226080 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae52633-0e3d-4857-85e3-efdce55483d1 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:49.226364 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae52633-0e3d-4857-85e3-efdce55483d1 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:49.226639 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae52633-0e3d-4857-85e3-efdce55483d1 None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:49.227847 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae52633-0e3d-4857-85e3-efdce55483d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:49.228306 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae52633-0e3d-4857-85e3-efdce55483d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:49.230120 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae52633-0e3d-4857-85e3-efdce55483d1 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:49.233463 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8fa522-ccdd-4fab-aaa4-6027d921b2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:49.234303 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fa522-ccdd-4fab-aaa4-6027d921b2e7 None admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:49.234758 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fa522-ccdd-4fab-aaa4-6027d921b2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:49.235204 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fa522-ccdd-4fab-aaa4-6027d921b2e7 None admin] PATH_INFO: `/v3/services` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:49.236008 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8fa522-ccdd-4fab-aaa4-6027d921b2e7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:49.237609 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8fa522-ccdd-4fab-aaa4-6027d921b2e7 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:49.241896 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 497/985] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:35:49.248516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae52633-0e3d-4857-85e3-efdce55483d1 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:49.273339 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 489/986] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 10 22:35:49 2022] POST /identity/v3/services => generated 242 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:35:49.407801 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:49.413508 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae52633-0e3d-4857-85e3-efdce55483d1 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:49.481973 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2538151-8955-45b6-9515-65b1dd786429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:49.482544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2538151-8955-45b6-9515-65b1dd786429 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:49.482821 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2538151-8955-45b6-9515-65b1dd786429 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:49.483091 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2538151-8955-45b6-9515-65b1dd786429 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:49.490111 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2538151-8955-45b6-9515-65b1dd786429 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:49.491563 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2538151-8955-45b6-9515-65b1dd786429 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:49.503274 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-d2538151-8955-45b6-9515-65b1dd786429 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:49.503274 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:49.503274 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:49.503274 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:49.503274 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:49.503274 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:49.503274 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:49.503274 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:49.503274 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:49.503274 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:49.503274 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:49.503274 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:49.503274 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:35:49.503274 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:35:49.503274 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:35:49.504188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:35:49.504188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:49.504188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:49.504188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:49.504188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:49.504188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:49.504188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:49.504188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:49.504188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:49.504188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:49.504188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:49.504188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:49.504188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:49.504188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:49.504188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:49.504188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:49.504188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:49.505128 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:35:49.505128 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:35:49.505128 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:35:49.505128 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:35:49.505128 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:35:49.505128 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:35:49.505128 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:49.505128 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:49.505128 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 498/987] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:35:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:49.511007 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8fa522-ccdd-4fab-aaa4-6027d921b2e7 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:49.617479 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8fa522-ccdd-4fab-aaa4-6027d921b2e7 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:49.654527 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b8897-3160-4398-b87d-5ef664671e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:49.655161 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b8897-3160-4398-b87d-5ef664671e6f None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:49.655444 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b8897-3160-4398-b87d-5ef664671e6f None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:49.655719 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b8897-3160-4398-b87d-5ef664671e6f None admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:49.656427 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966b8897-3160-4398-b87d-5ef664671e6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:49.656734 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966b8897-3160-4398-b87d-5ef664671e6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:49.658219 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966b8897-3160-4398-b87d-5ef664671e6f None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:49.677623 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 490/988] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:35:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:49.678943 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2538151-8955-45b6-9515-65b1dd786429 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:49.783192 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2538151-8955-45b6-9515-65b1dd786429 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:49.819466 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb4ed9-5e87-4f5f-94ea-f26977f1ad60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:49.820082 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb4ed9-5e87-4f5f-94ea-f26977f1ad60 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:49.820363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb4ed9-5e87-4f5f-94ea-f26977f1ad60 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:49.820638 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb4ed9-5e87-4f5f-94ea-f26977f1ad60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:49.821436 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb4ed9-5e87-4f5f-94ea-f26977f1ad60 None admin] RBAC: Adding query filter params (role.id=ee973ce49dc142a6b0763794e078333d, scope.project.id=3caa2ea846f7449b8c483788fc5846c8, user.id=aff5e804c64345b298c745287a1d0a32) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:49.821749 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb4ed9-5e87-4f5f-94ea-f26977f1ad60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:49.823190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb4ed9-5e87-4f5f-94ea-f26977f1ad60 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:49.837552 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 499/989] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:35:49 2022] GET /identity/v3/role_assignments?user.id=aff5e804c64345b298c745287a1d0a32&scope.project.id=3caa2ea846f7449b8c483788fc5846c8&role.id=ee973ce49dc142a6b0763794e078333d => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:50.566736 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed271b33-d011-486c-b8da-cd7a67bf3da5 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:50.567278 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed271b33-d011-486c-b8da-cd7a67bf3da5 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:50.567278 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed271b33-d011-486c-b8da-cd7a67bf3da5 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:50.568653 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 491/990] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:50.574013 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c5074-a615-47d7-a515-d07341104026 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:50.574408 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c5074-a615-47d7-a515-d07341104026 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:50.574568 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c5074-a615-47d7-a515-d07341104026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:50.593470 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-f36c5074-a615-47d7-a515-d07341104026 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:50.594664 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-f36c5074-a615-47d7-a515-d07341104026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:50.610432 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:50.666165 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 500/991] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:50 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:35:50.689703 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed271b33-d011-486c-b8da-cd7a67bf3da5 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:50.724591 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:50.794381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed271b33-d011-486c-b8da-cd7a67bf3da5 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:50.831923 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674e291-881e-4250-b592-5cea180e1252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:50.832547 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674e291-881e-4250-b592-5cea180e1252 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:50.832820 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674e291-881e-4250-b592-5cea180e1252 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:50.833085 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674e291-881e-4250-b592-5cea180e1252 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:50.833699 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8674e291-881e-4250-b592-5cea180e1252 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:50.835051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8674e291-881e-4250-b592-5cea180e1252 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:50.844740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-8674e291-881e-4250-b592-5cea180e1252 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 22:35:50.844740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:50.844740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:35:50.844740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:35:50.844740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:35:50.844740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:35:50.844740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:35:50.844740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:50.844740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 22:35:50.844740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:50.844740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:35:50.844740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:50.844740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:50.844740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:50.844740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:50.844740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:50.844740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:50.846026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:50.846026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:50.846026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:50.846026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:50.846026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:50.846026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:50.846026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:35:50.846026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:35:50.846026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:35:50.846026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:35:50.846026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:50.846026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:50.846026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:50.846026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:50.846026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:50.846026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:50.846883 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:50.846883 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:50.846883 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:50.846883 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:50.846883 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:50.846883 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:50.846883 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:50.846883 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:50.846883 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:50.846883 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:50.846883 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:35:50.846883 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:50.846883 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 22:35:50.846883 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:50.848213 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 492/992] 10.222.0.76 () {56 vars in 1126 bytes} [Wed Aug 10 22:35:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:50.851838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c5074-a615-47d7-a515-d07341104026 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:50.954001 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c5074-a615-47d7-a515-d07341104026 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:50.993711 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8996e28a-08d5-484c-b1a2-efcc6cdb06ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:50.994349 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996e28a-08d5-484c-b1a2-efcc6cdb06ec None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:50.994624 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996e28a-08d5-484c-b1a2-efcc6cdb06ec None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:50.994892 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996e28a-08d5-484c-b1a2-efcc6cdb06ec None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:50.995533 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8996e28a-08d5-484c-b1a2-efcc6cdb06ec None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:50.995832 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8996e28a-08d5-484c-b1a2-efcc6cdb06ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:50.997324 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8996e28a-08d5-484c-b1a2-efcc6cdb06ec None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:51.007349 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 501/993] 10.222.0.76 () {56 vars in 1122 bytes} [Wed Aug 10 22:35:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:51.008472 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674e291-881e-4250-b592-5cea180e1252 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:51.107887 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:51.112786 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674e291-881e-4250-b592-5cea180e1252 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:51.147561 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef620e0a-00f5-4b6b-8b4a-0d444fd0d55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:51.148163 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef620e0a-00f5-4b6b-8b4a-0d444fd0d55c None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:51.148458 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef620e0a-00f5-4b6b-8b4a-0d444fd0d55c None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:51.148777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef620e0a-00f5-4b6b-8b4a-0d444fd0d55c None admin] PATH_INFO: `/v3/services` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:51.149439 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef620e0a-00f5-4b6b-8b4a-0d444fd0d55c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:51.149738 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef620e0a-00f5-4b6b-8b4a-0d444fd0d55c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:51.151085 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef620e0a-00f5-4b6b-8b4a-0d444fd0d55c None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:51.156173 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 493/994] 10.222.0.76 () {56 vars in 1122 bytes} [Wed Aug 10 22:35:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:35:51.161706 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8996e28a-08d5-484c-b1a2-efcc6cdb06ec None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:51.178745 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:51.266018 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8996e28a-08d5-484c-b1a2-efcc6cdb06ec None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:51.299938 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a44e74-850f-4e94-8a05-049a5281429d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:51.300555 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a44e74-850f-4e94-8a05-049a5281429d None admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:51.300835 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a44e74-850f-4e94-8a05-049a5281429d None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:51.301178 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a44e74-850f-4e94-8a05-049a5281429d None admin] PATH_INFO: `/v3/endpoints` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:51.301855 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a44e74-850f-4e94-8a05-049a5281429d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:51.303222 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a44e74-850f-4e94-8a05-049a5281429d None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:51.325641 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 502/995] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:35:51 2022] POST /identity/v3/endpoints => generated 325 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:35:51.396731 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56036f9c-afe6-4efe-9fa0-bd1ef073932d None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:51.397082 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56036f9c-afe6-4efe-9fa0-bd1ef073932d None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:51.397312 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56036f9c-afe6-4efe-9fa0-bd1ef073932d None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:51.399059 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 494/996] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:51.404773 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6182724-1770-4f07-ac2d-f8d10d00489e None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:51.405069 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6182724-1770-4f07-ac2d-f8d10d00489e None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:51.405457 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6182724-1770-4f07-ac2d-f8d10d00489e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:51.428124 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-b6182724-1770-4f07-ac2d-f8d10d00489e None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:51.428124 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-b6182724-1770-4f07-ac2d-f8d10d00489e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:51.505791 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:51.513184 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 503/997] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:51 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:35:51.536840 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56036f9c-afe6-4efe-9fa0-bd1ef073932d None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:51.644363 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56036f9c-afe6-4efe-9fa0-bd1ef073932d None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:51.677917 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac4082-401c-4802-b768-2262783485f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:51.678547 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac4082-401c-4802-b768-2262783485f6 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:51.678820 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac4082-401c-4802-b768-2262783485f6 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:51.679089 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac4082-401c-4802-b768-2262783485f6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:51.680189 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac4082-401c-4802-b768-2262783485f6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:51.681660 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac4082-401c-4802-b768-2262783485f6 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:51.690904 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-42ac4082-401c-4802-b768-2262783485f6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 22:35:51.690904 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:51.690904 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:35:51.690904 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:35:51.690904 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:35:51.690904 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:35:51.690904 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:35:51.690904 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:51.690904 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 22:35:51.690904 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:51.690904 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:35:51.690904 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:51.690904 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:51.690904 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:51.690904 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:51.690904 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:51.690904 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:51.692169 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:51.692169 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:51.692169 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:51.692169 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:51.692169 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:51.692169 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:51.692169 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:35:51.692169 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:35:51.692169 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:35:51.692169 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:35:51.692169 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:51.692169 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:51.692169 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:51.692169 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:51.692169 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:51.692169 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:51.693499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:51.693499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:51.693499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:51.693499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:51.693499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:51.693499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:51.693499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:51.693499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:51.693499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:51.693499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:51.693499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:35:51.693499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:51.693499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 22:35:51.693499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:51.701316 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 495/998] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:35:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:51.701477 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6182724-1770-4f07-ac2d-f8d10d00489e None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:51.823815 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6182724-1770-4f07-ac2d-f8d10d00489e None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:51.859099 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7217fde-fe87-4496-a810-dc007e30c83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:51.859721 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7217fde-fe87-4496-a810-dc007e30c83c None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:51.860002 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7217fde-fe87-4496-a810-dc007e30c83c None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:51.860277 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7217fde-fe87-4496-a810-dc007e30c83c None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:51.860907 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7217fde-fe87-4496-a810-dc007e30c83c None admin] RBAC: Adding query filter params (name=compute) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:51.861212 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7217fde-fe87-4496-a810-dc007e30c83c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:51.862725 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7217fde-fe87-4496-a810-dc007e30c83c None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:51.872624 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 504/999] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 10 22:35:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:51.873956 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac4082-401c-4802-b768-2262783485f6 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:51.903807 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:51.944252 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a351e-38e1-4c91-b8ce-cae3b0aea444 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:51.944569 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a351e-38e1-4c91-b8ce-cae3b0aea444 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:51.944806 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a351e-38e1-4c91-b8ce-cae3b0aea444 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:51.946919 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 505/1000] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:35:51.951586 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67540a89-b707-4022-a81b-1c52a3609236 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:51.951891 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67540a89-b707-4022-a81b-1c52a3609236 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:51.952175 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67540a89-b707-4022-a81b-1c52a3609236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:51.971026 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-67540a89-b707-4022-a81b-1c52a3609236 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:51.972234 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-67540a89-b707-4022-a81b-1c52a3609236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:51.988912 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac4082-401c-4802-b768-2262783485f6 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:52.026091 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21f55d-69be-4a0b-9424-cde6579eee8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:52.026732 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21f55d-69be-4a0b-9424-cde6579eee8a None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:52.026982 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21f55d-69be-4a0b-9424-cde6579eee8a None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:52.027260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21f55d-69be-4a0b-9424-cde6579eee8a None admin] PATH_INFO: `/v3/services` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:52.027897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea21f55d-69be-4a0b-9424-cde6579eee8a None admin] RBAC: Adding query filter params (type=compute) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:52.028203 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea21f55d-69be-4a0b-9424-cde6579eee8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:52.029602 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea21f55d-69be-4a0b-9424-cde6579eee8a None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:52.039846 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 496/1001] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 10 22:35:51 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:52.040960 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21f55d-69be-4a0b-9424-cde6579eee8a None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:52.044020 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:52.057322 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 506/1002] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:51 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:35:52.084697 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67540a89-b707-4022-a81b-1c52a3609236 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:52.154377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:52.165398 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21f55d-69be-4a0b-9424-cde6579eee8a None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:52.199803 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67540a89-b707-4022-a81b-1c52a3609236 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:52.201851 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48553c17-332a-4760-a0c1-fda1e49f2fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:52.202471 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48553c17-332a-4760-a0c1-fda1e49f2fcd None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:52.202750 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48553c17-332a-4760-a0c1-fda1e49f2fcd None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:52.203024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48553c17-332a-4760-a0c1-fda1e49f2fcd None admin] PATH_INFO: `/v3/endpoints` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:52.203719 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48553c17-332a-4760-a0c1-fda1e49f2fcd None admin] RBAC: Adding query filter params (interface=public, service_id=084e85e3110b481cb70467daa9094295, region_id=RegionOne) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:52.204025 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48553c17-332a-4760-a0c1-fda1e49f2fcd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:52.205419 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48553c17-332a-4760-a0c1-fda1e49f2fcd None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:52.215524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 497/1003] 10.222.0.76 () {56 vars in 1251 bytes} [Wed Aug 10 22:35:52 2022] GET /identity/v3/endpoints?service_id=084e85e3110b481cb70467daa9094295&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:52.216540 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48553c17-332a-4760-a0c1-fda1e49f2fcd None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:52.242060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9194b552-9015-4790-b93e-e3a9d0eb184f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:52.242976 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194b552-9015-4790-b93e-e3a9d0eb184f None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:52.243260 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194b552-9015-4790-b93e-e3a9d0eb184f None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:52.243539 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194b552-9015-4790-b93e-e3a9d0eb184f None admin] PATH_INFO: `/v3/services/image` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:52.244122 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9194b552-9015-4790-b93e-e3a9d0eb184f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:52.245523 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9194b552-9015-4790-b93e-e3a9d0eb184f None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:52.254080 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-9194b552-9015-4790-b93e-e3a9d0eb184f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 22:35:52.254080 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:52.254080 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:35:52.254080 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:35:52.254080 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:35:52.254080 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:35:52.254080 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:35:52.254080 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:52.254080 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 22:35:52.254080 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:52.254080 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:35:52.254080 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:52.254080 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:52.254080 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:52.254080 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:52.254080 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:52.254080 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:52.255299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:52.255299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:52.255299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:52.255299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:52.255299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:52.255299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:52.255299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:35:52.255299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:35:52.255299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:35:52.255299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:35:52.255299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:52.255299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:52.255299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:52.255299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:52.255299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:52.255299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:52.256352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:52.256352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:52.256352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:52.256352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:52.256352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:52.256352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:52.256352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:52.256352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:52.256352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:52.256352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:52.256352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:35:52.256352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:52.256352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 22:35:52.256352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:52.261148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 507/1004] 10.222.0.76 () {56 vars in 1105 bytes} [Wed Aug 10 22:35:52 2022] GET /identity/v3/services/image => generated 86 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:35:52.262022 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9194b552-9015-4790-b93e-e3a9d0eb184f None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:52.325910 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48553c17-332a-4760-a0c1-fda1e49f2fcd None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:52.326895 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:52.365130 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a15084-47d0-4806-ad4a-f5556450e911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:52.365693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a15084-47d0-4806-ad4a-f5556450e911 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:52.365972 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a15084-47d0-4806-ad4a-f5556450e911 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:52.366244 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a15084-47d0-4806-ad4a-f5556450e911 None admin] PATH_INFO: `/v3/services` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:52.367624 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a15084-47d0-4806-ad4a-f5556450e911 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:52.369505 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a15084-47d0-4806-ad4a-f5556450e911 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:52.373998 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9194b552-9015-4790-b93e-e3a9d0eb184f None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:52.376540 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 498/1005] 10.222.0.76 () {56 vars in 1087 bytes} [Wed Aug 10 22:35:52 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 22:35:52.414277 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8134ce30-500f-487c-9ff9-0804eec9ab83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:52.417319 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8134ce30-500f-487c-9ff9-0804eec9ab83 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:52.417319 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8134ce30-500f-487c-9ff9-0804eec9ab83 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:52.417319 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8134ce30-500f-487c-9ff9-0804eec9ab83 None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:52.420819 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8134ce30-500f-487c-9ff9-0804eec9ab83 None admin] RBAC: Adding query filter params (name=image) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:52.420819 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8134ce30-500f-487c-9ff9-0804eec9ab83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:52.420819 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8134ce30-500f-487c-9ff9-0804eec9ab83 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:52.427454 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 508/1006] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:52 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:52.434334 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a15084-47d0-4806-ad4a-f5556450e911 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:52.457396 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:52.551778 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a15084-47d0-4806-ad4a-f5556450e911 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:52.588171 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e110f453-05f6-4d28-9f6d-841f6f536116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:52.588788 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110f453-05f6-4d28-9f6d-841f6f536116 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:52.589070 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110f453-05f6-4d28-9f6d-841f6f536116 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:52.589378 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110f453-05f6-4d28-9f6d-841f6f536116 None admin] PATH_INFO: `/v3/services` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:52.590007 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e110f453-05f6-4d28-9f6d-841f6f536116 None admin] RBAC: Adding query filter params (type=image) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:52.590308 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e110f453-05f6-4d28-9f6d-841f6f536116 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:52.591668 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e110f453-05f6-4d28-9f6d-841f6f536116 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:52.601763 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 499/1007] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:52 2022] GET /identity/v3/services?type=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:53.396356 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a8132-6dc4-4c97-98ed-274e606623b1 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:53.396885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a8132-6dc4-4c97-98ed-274e606623b1 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:53.396885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a8132-6dc4-4c97-98ed-274e606623b1 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:53.398717 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 509/1008] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:53.404110 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7887f7-d4b3-437b-8a9a-a31e9a61c0d4 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:53.405138 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7887f7-d4b3-437b-8a9a-a31e9a61c0d4 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:53.405138 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7887f7-d4b3-437b-8a9a-a31e9a61c0d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:53.424871 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-3e7887f7-d4b3-437b-8a9a-a31e9a61c0d4 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:53.428204 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-3e7887f7-d4b3-437b-8a9a-a31e9a61c0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:53.499411 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 500/1009] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:53 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:35:53.523023 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a8132-6dc4-4c97-98ed-274e606623b1 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:53.631824 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a8132-6dc4-4c97-98ed-274e606623b1 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:53.654508 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:53.667986 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089e1bc-8c7d-450b-b2c4-0e87619531f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:53.668619 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089e1bc-8c7d-450b-b2c4-0e87619531f6 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:53.668897 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089e1bc-8c7d-450b-b2c4-0e87619531f6 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:53.669168 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089e1bc-8c7d-450b-b2c4-0e87619531f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:53.670312 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b089e1bc-8c7d-450b-b2c4-0e87619531f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:53.671734 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b089e1bc-8c7d-450b-b2c4-0e87619531f6 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:53.677970 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 510/1010] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:53.679700 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7887f7-d4b3-437b-8a9a-a31e9a61c0d4 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:53.696728 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:53.792422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7887f7-d4b3-437b-8a9a-a31e9a61c0d4 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:53.826968 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aebd61-5e9f-4be5-b973-33ba985310d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:53.827576 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aebd61-5e9f-4be5-b973-33ba985310d7 None admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:53.827854 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aebd61-5e9f-4be5-b973-33ba985310d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:53.828126 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aebd61-5e9f-4be5-b973-33ba985310d7 None admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:53.828768 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aebd61-5e9f-4be5-b973-33ba985310d7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:53.830217 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aebd61-5e9f-4be5-b973-33ba985310d7 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:53.859235 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 501/1011] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 10 22:35:53 2022] POST /identity/v3/projects => generated 311 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:35:54.486626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54204613-3a61-4e5e-8209-912d24af4bd5 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:54.487164 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54204613-3a61-4e5e-8209-912d24af4bd5 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:54.487164 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54204613-3a61-4e5e-8209-912d24af4bd5 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:54.488528 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 511/1012] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:54.494723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9edba-7bc8-49d7-b363-47850a056437 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:54.495191 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9edba-7bc8-49d7-b363-47850a056437 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:54.495269 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9edba-7bc8-49d7-b363-47850a056437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:54.514657 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-0cf9edba-7bc8-49d7-b363-47850a056437 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:54.515869 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-0cf9edba-7bc8-49d7-b363-47850a056437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:54.578396 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:54.593308 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 502/1013] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:54 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:35:54.612314 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54204613-3a61-4e5e-8209-912d24af4bd5 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:54.644638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6c528-07af-41d6-a023-a1c1e4058f79 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:54.645001 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6c528-07af-41d6-a023-a1c1e4058f79 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:54.645179 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6c528-07af-41d6-a023-a1c1e4058f79 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:54.646884 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 503/1014] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:35:54.652131 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612b9e0-785b-46fd-9d95-0d87951d0bef None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:54.653164 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612b9e0-785b-46fd-9d95-0d87951d0bef None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:54.653164 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612b9e0-785b-46fd-9d95-0d87951d0bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:54.670556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-2612b9e0-785b-46fd-9d95-0d87951d0bef None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:54.671754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-2612b9e0-785b-46fd-9d95-0d87951d0bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:54.724690 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54204613-3a61-4e5e-8209-912d24af4bd5 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:54.744183 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 504/1015] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:54 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:35:54.764045 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ac1e6-520c-47bc-9084-06667ecfce89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:54.764698 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ac1e6-520c-47bc-9084-06667ecfce89 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:54.764960 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ac1e6-520c-47bc-9084-06667ecfce89 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:54.765243 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ac1e6-520c-47bc-9084-06667ecfce89 None admin] PATH_INFO: `/v3/services/compute` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:54.765925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ac1e6-520c-47bc-9084-06667ecfce89 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:54.766241 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612b9e0-785b-46fd-9d95-0d87951d0bef None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:54.767293 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ac1e6-520c-47bc-9084-06667ecfce89 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:54.778356 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-858ac1e6-520c-47bc-9084-06667ecfce89 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 22:35:54.778356 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:54.778356 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:35:54.778356 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:35:54.778356 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:35:54.778356 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:35:54.778356 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:35:54.778356 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:54.778356 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 22:35:54.778356 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:54.778356 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:35:54.778356 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:54.778356 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:54.778356 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:54.778356 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:54.778356 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:54.778356 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:54.779411 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:54.779411 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:54.779411 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:54.779411 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:54.779411 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:54.779411 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:54.779411 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:35:54.779411 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:35:54.779411 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:35:54.779411 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:35:54.779411 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:54.779411 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:54.779411 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:54.779411 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:54.779411 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:54.779411 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:54.780302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:54.780302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:54.780302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:54.780302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:54.780302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:54.780302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:54.780302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:54.780302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:54.780302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:54.780302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:54.780302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:35:54.780302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:54.780302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 22:35:54.780302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:54.781877 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 512/1016] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:35:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:54.785732 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ac1e6-520c-47bc-9084-06667ecfce89 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:54.845404 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:54.881176 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612b9e0-785b-46fd-9d95-0d87951d0bef None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:54.891645 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ac1e6-520c-47bc-9084-06667ecfce89 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:54.907735 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:54.921341 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e8fe5d-7e55-498e-a64a-4ae85d57609b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:54.922068 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8fe5d-7e55-498e-a64a-4ae85d57609b None admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:54.922349 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8fe5d-7e55-498e-a64a-4ae85d57609b None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:54.922626 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8fe5d-7e55-498e-a64a-4ae85d57609b None admin] PATH_INFO: `/v3/services` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:54.923264 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e8fe5d-7e55-498e-a64a-4ae85d57609b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:54.924630 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e8fe5d-7e55-498e-a64a-4ae85d57609b None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:54.926809 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e14d9c-72e7-4c9d-ab8e-970614a69fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:54.927422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e14d9c-72e7-4c9d-ab8e-970614a69fcf None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:54.927704 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e14d9c-72e7-4c9d-ab8e-970614a69fcf None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:54.927988 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e14d9c-72e7-4c9d-ab8e-970614a69fcf None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:54.928774 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e14d9c-72e7-4c9d-ab8e-970614a69fcf None admin] RBAC: Adding query filter params (name=compute) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:54.929259 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e14d9c-72e7-4c9d-ab8e-970614a69fcf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:54.930725 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e14d9c-72e7-4c9d-ab8e-970614a69fcf None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:54.936652 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 513/1017] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 10 22:35:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:54.943070 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e14d9c-72e7-4c9d-ab8e-970614a69fcf None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:54.944780 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 505/1018] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 10 22:35:54 2022] POST /identity/v3/services => generated 242 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:35:55.069655 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e14d9c-72e7-4c9d-ab8e-970614a69fcf None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:55.088748 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:55.114148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2028188a-2c87-4a18-8780-f7def9801a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:55.114759 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028188a-2c87-4a18-8780-f7def9801a64 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:55.115042 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028188a-2c87-4a18-8780-f7def9801a64 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:55.115315 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028188a-2c87-4a18-8780-f7def9801a64 None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:55.115991 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2028188a-2c87-4a18-8780-f7def9801a64 None admin] RBAC: Adding query filter params (type=compute) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:55.116295 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2028188a-2c87-4a18-8780-f7def9801a64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:55.117698 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2028188a-2c87-4a18-8780-f7def9801a64 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:55.123670 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 514/1019] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 10 22:35:54 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:35:55.129584 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e8fe5d-7e55-498e-a64a-4ae85d57609b None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:55.238451 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e8fe5d-7e55-498e-a64a-4ae85d57609b None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:55.275501 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096808da-aac2-4dae-8e17-c557c4f44628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:55.276535 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096808da-aac2-4dae-8e17-c557c4f44628 None admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:55.277022 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096808da-aac2-4dae-8e17-c557c4f44628 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:55.277565 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096808da-aac2-4dae-8e17-c557c4f44628 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:55.278424 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096808da-aac2-4dae-8e17-c557c4f44628 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:55.279990 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096808da-aac2-4dae-8e17-c557c4f44628 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:55.305082 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 506/1020] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:35:55 2022] POST /identity/v3/endpoints => generated 334 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:35:55.943049 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a997f-52af-41c6-ac02-0ea48bc0c9fe None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:55.943602 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a997f-52af-41c6-ac02-0ea48bc0c9fe None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:55.943602 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a997f-52af-41c6-ac02-0ea48bc0c9fe None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:55.945901 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 515/1021] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:55.950850 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176b07b-9660-4e35-bd7a-2d2291894f20 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:55.951242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176b07b-9660-4e35-bd7a-2d2291894f20 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:55.951419 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176b07b-9660-4e35-bd7a-2d2291894f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:55.969733 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-7176b07b-9660-4e35-bd7a-2d2291894f20 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:55.970983 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-7176b07b-9660-4e35-bd7a-2d2291894f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:56.007641 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:56.051470 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 507/1022] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:35:56.089712 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a997f-52af-41c6-ac02-0ea48bc0c9fe None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:56.209191 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a997f-52af-41c6-ac02-0ea48bc0c9fe None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:56.243245 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea489c3-0a5b-40fc-8791-c875f9770539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:56.243876 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea489c3-0a5b-40fc-8791-c875f9770539 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:56.244155 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea489c3-0a5b-40fc-8791-c875f9770539 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:56.244427 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea489c3-0a5b-40fc-8791-c875f9770539 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:56.245618 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea489c3-0a5b-40fc-8791-c875f9770539 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:56.247054 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea489c3-0a5b-40fc-8791-c875f9770539 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:56.248770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 516/1023] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:56.254463 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176b07b-9660-4e35-bd7a-2d2291894f20 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:56.354532 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176b07b-9660-4e35-bd7a-2d2291894f20 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:56.387896 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c8a979-7590-4286-a1ed-20e492d512bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:56.388497 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8a979-7590-4286-a1ed-20e492d512bc None admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:56.388774 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8a979-7590-4286-a1ed-20e492d512bc None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:56.389046 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8a979-7590-4286-a1ed-20e492d512bc None admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:56.389706 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c8a979-7590-4286-a1ed-20e492d512bc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:56.391162 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c8a979-7590-4286-a1ed-20e492d512bc None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:56.417498 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 508/1024] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 10 22:35:56 2022] POST /identity/v3/users => generated 285 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:35:57.027167 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2efd8-7c1c-4e0b-8b4a-f7aa6679213d None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:57.027694 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2efd8-7c1c-4e0b-8b4a-f7aa6679213d None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:57.027694 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2efd8-7c1c-4e0b-8b4a-f7aa6679213d None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:57.029176 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 517/1025] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:35:57.034264 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0289f27-63ec-4dab-b615-151677206c47 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:57.034671 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0289f27-63ec-4dab-b615-151677206c47 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:57.034856 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0289f27-63ec-4dab-b615-151677206c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:57.054260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-d0289f27-63ec-4dab-b615-151677206c47 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:57.055434 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-d0289f27-63ec-4dab-b615-151677206c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:57.072697 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:57.126979 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 509/1026] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:35:57.149603 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2efd8-7c1c-4e0b-8b4a-f7aa6679213d None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:57.201270 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:57.258009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2efd8-7c1c-4e0b-8b4a-f7aa6679213d None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:57.292291 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c6ba97-6d12-4e8a-8092-82c0f6bf56c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:57.292846 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c6ba97-6d12-4e8a-8092-82c0f6bf56c4 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:57.293126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c6ba97-6d12-4e8a-8092-82c0f6bf56c4 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:57.293423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c6ba97-6d12-4e8a-8092-82c0f6bf56c4 None admin] PATH_INFO: `/v3/services/image` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:57.293999 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c6ba97-6d12-4e8a-8092-82c0f6bf56c4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:57.295368 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c6ba97-6d12-4e8a-8092-82c0f6bf56c4 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:57.304596 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-46c6ba97-6d12-4e8a-8092-82c0f6bf56c4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 22:35:57.304596 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:57.304596 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:35:57.304596 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:35:57.304596 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:35:57.304596 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:35:57.304596 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:35:57.304596 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:57.304596 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 22:35:57.304596 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:57.304596 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:35:57.304596 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:57.304596 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:57.304596 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:57.304596 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:57.304596 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:57.304596 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:57.305888 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:57.305888 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:57.305888 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:57.305888 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:57.305888 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:57.305888 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:57.305888 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:35:57.305888 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:35:57.305888 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:35:57.305888 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:35:57.305888 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:57.305888 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:57.305888 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:57.305888 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:57.305888 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:57.305888 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:57.306723 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:57.306723 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:57.306723 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:57.306723 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:57.306723 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:57.306723 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:57.306723 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:57.306723 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:57.306723 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:57.306723 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:57.306723 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:35:57.306723 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:35:57.306723 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 22:35:57.306723 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:57.309587 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 518/1027] 10.222.0.76 () {56 vars in 1105 bytes} [Wed Aug 10 22:35:57 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:57.311657 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0289f27-63ec-4dab-b615-151677206c47 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:57.411506 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0289f27-63ec-4dab-b615-151677206c47 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:57.455223 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2746cd87-0e51-4740-b625-6f9cefb1766f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:57.455838 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746cd87-0e51-4740-b625-6f9cefb1766f None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:57.456116 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746cd87-0e51-4740-b625-6f9cefb1766f None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:57.456387 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746cd87-0e51-4740-b625-6f9cefb1766f None admin] PATH_INFO: `/v3/services` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:57.457145 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2746cd87-0e51-4740-b625-6f9cefb1766f None admin] RBAC: Adding query filter params (name=image) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:57.457474 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2746cd87-0e51-4740-b625-6f9cefb1766f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:57.458846 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2746cd87-0e51-4740-b625-6f9cefb1766f None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:57.469309 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 510/1028] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:57.470601 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c6ba97-6d12-4e8a-8092-82c0f6bf56c4 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:57.501087 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a618b3a-30c4-4db1-a74b-51c35177e7ab None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:57.501439 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a618b3a-30c4-4db1-a74b-51c35177e7ab None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:57.501685 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a618b3a-30c4-4db1-a74b-51c35177e7ab None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:57.503316 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 511/1029] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:35:57.510548 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e8221-5bdb-42a0-81e8-58c762c4cc4f None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:57.510829 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e8221-5bdb-42a0-81e8-58c762c4cc4f None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:57.511103 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e8221-5bdb-42a0-81e8-58c762c4cc4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:57.531446 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-cd8e8221-5bdb-42a0-81e8-58c762c4cc4f None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:57.532638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-cd8e8221-5bdb-42a0-81e8-58c762c4cc4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:57.546021 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:57.581418 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c6ba97-6d12-4e8a-8092-82c0f6bf56c4 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:57.596167 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:57.621448 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 512/1030] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:35:57.625215 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36abc0f-6e26-4d2a-b779-07435c818b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:57.625870 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36abc0f-6e26-4d2a-b779-07435c818b90 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:57.626149 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36abc0f-6e26-4d2a-b779-07435c818b90 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:57.626418 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36abc0f-6e26-4d2a-b779-07435c818b90 None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:57.627039 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36abc0f-6e26-4d2a-b779-07435c818b90 None admin] RBAC: Adding query filter params (type=image) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:57.627339 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36abc0f-6e26-4d2a-b779-07435c818b90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:57.628695 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36abc0f-6e26-4d2a-b779-07435c818b90 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:57.634386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 519/1031] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:57 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:57.638876 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e8221-5bdb-42a0-81e8-58c762c4cc4f None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:57.639934 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36abc0f-6e26-4d2a-b779-07435c818b90 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:57.748227 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36abc0f-6e26-4d2a-b779-07435c818b90 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:57.749770 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e8221-5bdb-42a0-81e8-58c762c4cc4f None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:57.785760 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97106052-9ab9-4cb8-8913-3a008680b292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:57.786204 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79830bcc-5d64-496a-a704-347beb7b063a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:57.786427 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97106052-9ab9-4cb8-8913-3a008680b292 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:57.786663 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97106052-9ab9-4cb8-8913-3a008680b292 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:57.786835 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79830bcc-5d64-496a-a704-347beb7b063a None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:57.786941 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97106052-9ab9-4cb8-8913-3a008680b292 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:57.787093 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79830bcc-5d64-496a-a704-347beb7b063a None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:57.787371 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79830bcc-5d64-496a-a704-347beb7b063a None admin] PATH_INFO: `/v3/endpoints` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:57.788696 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79830bcc-5d64-496a-a704-347beb7b063a None admin] RBAC: Adding query filter params (interface=public, service_id=1c82f0c3d7364a92b41dac8d3fe920ed, region_id=RegionOne) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:57.789101 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79830bcc-5d64-496a-a704-347beb7b063a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:57.790535 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79830bcc-5d64-496a-a704-347beb7b063a None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:57.800108 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 520/1032] 10.222.0.76 () {56 vars in 1251 bytes} [Wed Aug 10 22:35:57 2022] GET /identity/v3/endpoints?service_id=1c82f0c3d7364a92b41dac8d3fe920ed&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:57.801053 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79830bcc-5d64-496a-a704-347beb7b063a None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:57.801457 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97106052-9ab9-4cb8-8913-3a008680b292 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:35:57.806254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-97106052-9ab9-4cb8-8913-3a008680b292 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 22:35:57.806254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:57.806254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:57.806254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:57.806254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:57.806254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:57.806254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:57.806254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:57.806254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:57.806254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:57.806254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:57.806254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:57.806254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:35:57.806254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:35:57.806254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:35:57.806254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:35:57.807186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:35:57.807186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:35:57.807186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:35:57.807186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:35:57.807186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:57.807186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:57.807186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:57.807186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:57.807186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:57.807186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:57.807186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:57.807186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:57.807186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:57.807186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:57.807186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:57.807186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:57.808038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:57.808038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:57.808038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:57.808038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:57.808038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:35:57.808038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:35:57.808038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:35:57.808038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:35:57.808038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:35:57.808038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:35:57.808038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 22:35:57.808038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:57.808038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 513/1033] 10.222.0.76 () {56 vars in 1120 bytes} [Wed Aug 10 22:35:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:57.813390 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97106052-9ab9-4cb8-8913-3a008680b292 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:57.824380 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:57.851158 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:57.920362 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79830bcc-5d64-496a-a704-347beb7b063a None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:57.927585 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97106052-9ab9-4cb8-8913-3a008680b292 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:57.963455 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688682fd-2c82-4df9-b705-c205aa98d6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:57.964108 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688682fd-2c82-4df9-b705-c205aa98d6ba None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:57.964382 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688682fd-2c82-4df9-b705-c205aa98d6ba None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:57.964659 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688682fd-2c82-4df9-b705-c205aa98d6ba None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:57.965259 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688682fd-2c82-4df9-b705-c205aa98d6ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:57.966684 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688682fd-2c82-4df9-b705-c205aa98d6ba None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:57.968973 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eacdc3-04dc-42da-bf08-c9850ce4210d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:57.969857 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eacdc3-04dc-42da-bf08-c9850ce4210d None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:57.970231 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eacdc3-04dc-42da-bf08-c9850ce4210d None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:57.970602 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eacdc3-04dc-42da-bf08-c9850ce4210d None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:57.971354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eacdc3-04dc-42da-bf08-c9850ce4210d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:57.971660 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eacdc3-04dc-42da-bf08-c9850ce4210d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:57.973008 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eacdc3-04dc-42da-bf08-c9850ce4210d None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:57.974057 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 521/1034] 10.222.0.76 () {56 vars in 1087 bytes} [Wed Aug 10 22:35:57 2022] GET /identity/v3/services => generated 1957 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 22:35:57.988914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 514/1035] 10.222.0.76 () {56 vars in 1115 bytes} [Wed Aug 10 22:35:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:35:57.989849 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688682fd-2c82-4df9-b705-c205aa98d6ba None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:58.107121 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688682fd-2c82-4df9-b705-c205aa98d6ba None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:58.108813 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:58.150928 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a358a-dec3-402e-a4f5-8fad51fb4a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:58.151577 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a358a-dec3-402e-a4f5-8fad51fb4a77 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:58.151856 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a358a-dec3-402e-a4f5-8fad51fb4a77 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:58.152130 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a358a-dec3-402e-a4f5-8fad51fb4a77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:58.162754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86a358a-dec3-402e-a4f5-8fad51fb4a77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:58.164291 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86a358a-dec3-402e-a4f5-8fad51fb4a77 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:58.179357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-c86a358a-dec3-402e-a4f5-8fad51fb4a77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:58.179357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:58.179357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:58.179357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:58.179357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:58.179357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:58.179357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:58.179357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:58.179357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:58.179357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:58.179357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:58.179357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:58.179357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:58.179357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:58.179357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:58.179357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:58.180380 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:58.180380 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:58.180380 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:58.180380 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:58.180380 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:58.180380 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:58.180380 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:58.180380 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:58.180380 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:58.180380 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:58.180380 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:58.180380 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:58.180380 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:58.180380 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:58.180380 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:58.180380 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:58.180380 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:58.181322 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:58.181322 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:58.181322 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:58.181322 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 522/1036] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:35:58.186395 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eacdc3-04dc-42da-bf08-c9850ce4210d None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:58.257556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:58.292765 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eacdc3-04dc-42da-bf08-c9850ce4210d None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:58.327250 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd8dad-34fd-4ac2-b291-a843ab9927f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:58.327855 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd8dad-34fd-4ac2-b291-a843ab9927f8 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:58.328144 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd8dad-34fd-4ac2-b291-a843ab9927f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:58.328437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd8dad-34fd-4ac2-b291-a843ab9927f8 None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:58.329072 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecd8dad-34fd-4ac2-b291-a843ab9927f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:58.329395 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecd8dad-34fd-4ac2-b291-a843ab9927f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:58.330810 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecd8dad-34fd-4ac2-b291-a843ab9927f8 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:58.343618 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 515/1037] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:58.349011 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a358a-dec3-402e-a4f5-8fad51fb4a77 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:58.464015 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a358a-dec3-402e-a4f5-8fad51fb4a77 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:58.499559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d3e015-b05d-4957-8eba-2ce80269bb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:58.500194 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3e015-b05d-4957-8eba-2ce80269bb94 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:58.500473 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3e015-b05d-4957-8eba-2ce80269bb94 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:58.500743 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3e015-b05d-4957-8eba-2ce80269bb94 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:58.509362 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d3e015-b05d-4957-8eba-2ce80269bb94 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:58.510825 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d3e015-b05d-4957-8eba-2ce80269bb94 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:58.515889 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ccdbe5-75a4-4a23-8a97-7e550d273bdb None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:58.516179 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ccdbe5-75a4-4a23-8a97-7e550d273bdb None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:58.516454 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ccdbe5-75a4-4a23-8a97-7e550d273bdb None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:58.518001 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 516/1038] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:35:58.522036 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-a0d3e015-b05d-4957-8eba-2ce80269bb94 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:35:58.522036 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:58.522036 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:58.522036 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:58.522036 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:58.522036 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:58.522036 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:58.522036 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:58.522036 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:58.522036 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:58.522036 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:58.522036 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:58.522036 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:35:58.522036 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:35:58.522036 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:35:58.522036 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:35:58.523552 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:58.523552 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:58.523552 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:35:58.523552 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:58.523552 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:35:58.523552 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:35:58.523552 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:58.523552 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:58.523552 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:58.523552 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:58.523552 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:58.523552 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:58.523552 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:58.523552 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:58.523552 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:58.523552 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:58.523552 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:58.523552 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:58.525190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:58.525190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:58.525190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:35:58.525190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:35:58.525190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:35:58.525190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:35:58.525190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:35:58.525190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:35:58.525190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:35:58.525190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:58.525798 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec408776-8544-46f1-a8da-fe5cc78d1d9b None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:58.525798 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec408776-8544-46f1-a8da-fe5cc78d1d9b None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:58.525798 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec408776-8544-46f1-a8da-fe5cc78d1d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:58.525979 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 523/1039] 10.222.0.76 () {56 vars in 1093 bytes} [Wed Aug 10 22:35:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:35:58.529206 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d3e015-b05d-4957-8eba-2ce80269bb94 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:58.546832 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-ec408776-8544-46f1-a8da-fe5cc78d1d9b None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:35:58.547018 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:58.548055 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-ec408776-8544-46f1-a8da-fe5cc78d1d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:35:58.620029 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 517/1040] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:35:58 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Aug 10 22:35:58.639656 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d3e015-b05d-4957-8eba-2ce80269bb94 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:58.643245 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec408776-8544-46f1-a8da-fe5cc78d1d9b None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:58.676146 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc132fe-736a-43a8-b9e8-aea180f5344f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:58.676762 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc132fe-736a-43a8-b9e8-aea180f5344f None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:58.677042 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc132fe-736a-43a8-b9e8-aea180f5344f None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:58.677342 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc132fe-736a-43a8-b9e8-aea180f5344f None admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:58.678066 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc132fe-736a-43a8-b9e8-aea180f5344f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:58.678909 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc132fe-736a-43a8-b9e8-aea180f5344f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:58.680332 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc132fe-736a-43a8-b9e8-aea180f5344f None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:58.700435 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 524/1041] 10.222.0.76 () {56 vars in 1133 bytes} [Wed Aug 10 22:35:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:35:58.706000 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc132fe-736a-43a8-b9e8-aea180f5344f None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:58.714225 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:58.758368 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec408776-8544-46f1-a8da-fe5cc78d1d9b None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:58.790691 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:58.795782 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244d303-1510-4ead-90be-9d25afb1ba99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:58.796415 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244d303-1510-4ead-90be-9d25afb1ba99 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:58.796695 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244d303-1510-4ead-90be-9d25afb1ba99 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:58.796972 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244d303-1510-4ead-90be-9d25afb1ba99 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:58.808124 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8244d303-1510-4ead-90be-9d25afb1ba99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:35:58.813522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-8244d303-1510-4ead-90be-9d25afb1ba99 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:35:58.813522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:58.813522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:58.813522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:58.813522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:58.813522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:58.813522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:58.813522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:58.813522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:58.813522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:58.813522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:58.813522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:58.813522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:35:58.813522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:35:58.813522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:35:58.813522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:35:58.814536 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:35:58.814536 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:35:58.814536 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:35:58.814536 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:35:58.814536 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:58.814536 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:58.814536 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:58.814536 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:58.814536 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:58.814536 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:58.814536 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:58.814536 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:58.814536 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:58.814536 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:58.814536 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:58.814536 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:58.815369 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:58.815369 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:58.815369 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:58.815369 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:58.815369 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:35:58.815369 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:35:58.815369 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:35:58.815369 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:35:58.815369 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:35:58.815369 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:35:58.815369 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:35:58.815369 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:35:58.815369 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 518/1042] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:35:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:35:58.821220 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244d303-1510-4ead-90be-9d25afb1ba99 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:58.826536 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc132fe-736a-43a8-b9e8-aea180f5344f None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:58.863014 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d755e8a8-2b7a-43df-9e09-51a940b8fa58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:58.863931 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755e8a8-2b7a-43df-9e09-51a940b8fa58 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:58.864367 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755e8a8-2b7a-43df-9e09-51a940b8fa58 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:58.864879 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755e8a8-2b7a-43df-9e09-51a940b8fa58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:58.923993 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244d303-1510-4ead-90be-9d25afb1ba99 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:58.938414 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d755e8a8-2b7a-43df-9e09-51a940b8fa58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:58.939974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d755e8a8-2b7a-43df-9e09-51a940b8fa58 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:58.943688 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:58.964656 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86bfeb4-b1ff-421f-bb53-3b4098ba7206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:58.965185 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86bfeb4-b1ff-421f-bb53-3b4098ba7206 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:58.965512 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86bfeb4-b1ff-421f-bb53-3b4098ba7206 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:58.965765 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86bfeb4-b1ff-421f-bb53-3b4098ba7206 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:58.966425 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86bfeb4-b1ff-421f-bb53-3b4098ba7206 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:58.966716 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86bfeb4-b1ff-421f-bb53-3b4098ba7206 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:58.968101 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86bfeb4-b1ff-421f-bb53-3b4098ba7206 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:58.976781 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 519/1043] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:35:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:58.982966 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86bfeb4-b1ff-421f-bb53-3b4098ba7206 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:59.018844 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:59.051933 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 525/1044] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:35:58 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 22:35:59.059417 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d755e8a8-2b7a-43df-9e09-51a940b8fa58 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:59.096134 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86bfeb4-b1ff-421f-bb53-3b4098ba7206 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:59.134495 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0084bf81-8c7e-4c05-abed-7115020715ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:59.135136 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084bf81-8c7e-4c05-abed-7115020715ec None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:59.135419 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084bf81-8c7e-4c05-abed-7115020715ec None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:59.135693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084bf81-8c7e-4c05-abed-7115020715ec None admin] PATH_INFO: `/v3/users/6b051279581945c69892101879b2ad4e` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:59.152203 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0084bf81-8c7e-4c05-abed-7115020715ec None admin] RBAC: Authorizing `identity:get_user(user_id=6b051279581945c69892101879b2ad4e)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:59.154081 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0084bf81-8c7e-4c05-abed-7115020715ec None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:59.160277 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 520/1045] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:35:58 2022] GET /identity/v3/users/6b051279581945c69892101879b2ad4e => generated 285 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:59.161641 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0084bf81-8c7e-4c05-abed-7115020715ec None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:59.174850 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d755e8a8-2b7a-43df-9e09-51a940b8fa58 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:59.211079 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:59.216179 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f76dd4-ae17-44bf-b2bf-0c9e0fd0c4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:59.216969 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f76dd4-ae17-44bf-b2bf-0c9e0fd0c4bb None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:59.217261 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f76dd4-ae17-44bf-b2bf-0c9e0fd0c4bb None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:59.217574 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f76dd4-ae17-44bf-b2bf-0c9e0fd0c4bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:59.231101 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f76dd4-ae17-44bf-b2bf-0c9e0fd0c4bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:59.233058 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f76dd4-ae17-44bf-b2bf-0c9e0fd0c4bb None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:59.248359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-41f76dd4-ae17-44bf-b2bf-0c9e0fd0c4bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:59.248359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:59.248359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:59.248359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:59.248359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:59.248359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:59.248359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:59.248359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:59.248359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:59.248359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:59.248359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:59.248359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:59.248359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:35:59.248359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:35:59.248359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:35:59.248359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:35:59.249423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:59.249423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:59.249423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:59.249423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:59.249423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:59.249423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:59.249423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:59.249423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:59.249423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:59.249423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:59.249423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:59.249423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:59.249423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:59.249423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:59.249423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:59.249423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:59.249423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:35:59.255367 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:35:59.255367 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:35:59.255367 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:59.255367 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 526/1046] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:35:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:35:59.258040 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f76dd4-ae17-44bf-b2bf-0c9e0fd0c4bb None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:59.275719 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0084bf81-8c7e-4c05-abed-7115020715ec None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:59.284044 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:59.313186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b521338-7d80-4aa3-91cf-6e886a78bb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:59.313964 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b521338-7d80-4aa3-91cf-6e886a78bb96 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:59.314243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b521338-7d80-4aa3-91cf-6e886a78bb96 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:59.314514 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b521338-7d80-4aa3-91cf-6e886a78bb96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:59.363009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f76dd4-ae17-44bf-b2bf-0c9e0fd0c4bb None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:59.389543 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b521338-7d80-4aa3-91cf-6e886a78bb96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:59.390992 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b521338-7d80-4aa3-91cf-6e886a78bb96 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:59.402105 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51a6ad-b882-4837-b9fa-4ad05b1f3f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:59.403036 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51a6ad-b882-4837-b9fa-4ad05b1f3f78 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:59.403470 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51a6ad-b882-4837-b9fa-4ad05b1f3f78 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:59.403746 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51a6ad-b882-4837-b9fa-4ad05b1f3f78 None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:59.404389 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf51a6ad-b882-4837-b9fa-4ad05b1f3f78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:59.404697 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf51a6ad-b882-4837-b9fa-4ad05b1f3f78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:59.406102 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf51a6ad-b882-4837-b9fa-4ad05b1f3f78 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:59.425965 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 527/1047] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:35:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:35:59.427028 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51a6ad-b882-4837-b9fa-4ad05b1f3f78 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:59.440937 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:59.514019 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 521/1048] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:35:59 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 22:35:59.519371 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b521338-7d80-4aa3-91cf-6e886a78bb96 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:59.521403 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:59.539288 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51a6ad-b882-4837-b9fa-4ad05b1f3f78 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:59.576168 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a05a6c-b14c-4a41-bd34-cf62827065de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:59.576803 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a05a6c-b14c-4a41-bd34-cf62827065de None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:59.577084 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a05a6c-b14c-4a41-bd34-cf62827065de None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:59.577390 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a05a6c-b14c-4a41-bd34-cf62827065de None admin] PATH_INFO: `/v3/projects/service` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:59.585894 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a05a6c-b14c-4a41-bd34-cf62827065de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:59.587361 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a05a6c-b14c-4a41-bd34-cf62827065de None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:59.598900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-34a05a6c-b14c-4a41-bd34-cf62827065de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:59.598900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:35:59.598900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:35:59.598900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:35:59.598900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:35:59.598900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:35:59.598900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:35:59.598900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:35:59.598900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:35:59.598900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:35:59.598900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:35:59.598900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:35:59.598900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:35:59.598900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:35:59.598900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:35:59.599800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:35:59.599800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:35:59.599800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:35:59.599800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:35:59.599800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:35:59.599800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:35:59.599800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:35:59.599800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:35:59.599800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:35:59.599800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:35:59.599800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:35:59.599800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:35:59.599800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:35:59.599800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:35:59.599800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:35:59.599800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:35:59.599800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:35:59.606081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:35:59.606081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:35:59.606081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:35:59.606081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:35:59.606081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:35:59.606081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:35:59.606081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:35:59.606081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:35:59.606081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 528/1049] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:35:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:35:59.606565 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a05a6c-b14c-4a41-bd34-cf62827065de None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:59.631061 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b521338-7d80-4aa3-91cf-6e886a78bb96 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:59.673189 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321de8a5-0d51-4e74-bb49-04a0b3b83bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:59.674185 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321de8a5-0d51-4e74-bb49-04a0b3b83bfa None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:59.674647 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321de8a5-0d51-4e74-bb49-04a0b3b83bfa None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:59.675123 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321de8a5-0d51-4e74-bb49-04a0b3b83bfa None admin] PATH_INFO: `/v3/projects/9e62fda33d2843ca9cd1614d910b1fb9` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:59.676473 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321de8a5-0d51-4e74-bb49-04a0b3b83bfa None admin] RBAC: Authorizing `identity:get_project(project_id=9e62fda33d2843ca9cd1614d910b1fb9)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:59.678118 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321de8a5-0d51-4e74-bb49-04a0b3b83bfa None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:59.680146 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 522/1050] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:35:59 2022] GET /identity/v3/projects/9e62fda33d2843ca9cd1614d910b1fb9 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:35:59.685761 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321de8a5-0d51-4e74-bb49-04a0b3b83bfa None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:59.695407 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:59.733915 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a05a6c-b14c-4a41-bd34-cf62827065de None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:59.774651 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445fb9c-0745-4be6-9f6e-67824c47ab8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:59.775274 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445fb9c-0745-4be6-9f6e-67824c47ab8c None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:59.775492 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445fb9c-0745-4be6-9f6e-67824c47ab8c None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:59.775764 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445fb9c-0745-4be6-9f6e-67824c47ab8c None admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:59.776468 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445fb9c-0745-4be6-9f6e-67824c47ab8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:59.776778 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445fb9c-0745-4be6-9f6e-67824c47ab8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:59.780552 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445fb9c-0745-4be6-9f6e-67824c47ab8c None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:59.795238 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 529/1051] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:35:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:35:59.798521 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321de8a5-0d51-4e74-bb49-04a0b3b83bfa None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:59.802027 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445fb9c-0745-4be6-9f6e-67824c47ab8c None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:35:59.805893 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:35:59.835815 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19475c6-f041-4cf1-96e5-ac5202e46578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:59.836562 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19475c6-f041-4cf1-96e5-ac5202e46578 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:59.836839 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19475c6-f041-4cf1-96e5-ac5202e46578 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:59.837105 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19475c6-f041-4cf1-96e5-ac5202e46578 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:59.837849 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19475c6-f041-4cf1-96e5-ac5202e46578 None admin] RBAC: Adding query filter params (role.id=6641e95fdc1f4c098ebc372bb692d227, scope.project.id=9e62fda33d2843ca9cd1614d910b1fb9, user.id=6b051279581945c69892101879b2ad4e) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:59.838150 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19475c6-f041-4cf1-96e5-ac5202e46578 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:59.839565 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19475c6-f041-4cf1-96e5-ac5202e46578 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:59.851547 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 523/1052] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:35:59 2022] GET /identity/v3/role_assignments?user.id=6b051279581945c69892101879b2ad4e&scope.project.id=9e62fda33d2843ca9cd1614d910b1fb9&role.id=6641e95fdc1f4c098ebc372bb692d227 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:35:59.917242 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445fb9c-0745-4be6-9f6e-67824c47ab8c None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:35:59.954603 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a10e0a-1d18-490f-975d-66824f13f9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:35:59.955224 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a10e0a-1d18-490f-975d-66824f13f9ea None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:35:59.955504 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a10e0a-1d18-490f-975d-66824f13f9ea None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:35:59.955776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a10e0a-1d18-490f-975d-66824f13f9ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:35:59.956504 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a10e0a-1d18-490f-975d-66824f13f9ea None admin] RBAC: Adding query filter params (role.id=ee973ce49dc142a6b0763794e078333d, scope.project.id=3caa2ea846f7449b8c483788fc5846c8, user.id=b22838a8ad3949e59cdbaaa1801cd07f) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:35:59.956810 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a10e0a-1d18-490f-975d-66824f13f9ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:35:59.958317 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a10e0a-1d18-490f-975d-66824f13f9ea None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:35:59.964982 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 530/1053] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:35:59 2022] GET /identity/v3/role_assignments?user.id=b22838a8ad3949e59cdbaaa1801cd07f&scope.project.id=3caa2ea846f7449b8c483788fc5846c8&role.id=ee973ce49dc142a6b0763794e078333d => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:36:00.061801 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8d66c-126a-47f7-af6f-fb7234d2f10a None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:00.062106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8d66c-126a-47f7-af6f-fb7234d2f10a None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:00.062300 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8d66c-126a-47f7-af6f-fb7234d2f10a None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:00.064089 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 524/1054] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:36:00.069549 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b898326-47ce-4292-81f5-8b7e149905d2 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:00.069969 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b898326-47ce-4292-81f5-8b7e149905d2 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:00.070134 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b898326-47ce-4292-81f5-8b7e149905d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:00.089698 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-6b898326-47ce-4292-81f5-8b7e149905d2 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:36:00.090931 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-6b898326-47ce-4292-81f5-8b7e149905d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:36:00.169606 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:00.176420 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 531/1055] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:36:00 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:36:00.200198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8d66c-126a-47f7-af6f-fb7234d2f10a None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:00.275515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:00.309440 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8d66c-126a-47f7-af6f-fb7234d2f10a None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:00.344337 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2188ba-2f1b-4ae0-881e-ee5bbb4ad24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:00.344980 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2188ba-2f1b-4ae0-881e-ee5bbb4ad24f None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:00.345261 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2188ba-2f1b-4ae0-881e-ee5bbb4ad24f None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:00.345561 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2188ba-2f1b-4ae0-881e-ee5bbb4ad24f None admin] PATH_INFO: `/v3/services/image` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:00.346143 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2188ba-2f1b-4ae0-881e-ee5bbb4ad24f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:00.347503 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2188ba-2f1b-4ae0-881e-ee5bbb4ad24f None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:00.357055 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-4d2188ba-2f1b-4ae0-881e-ee5bbb4ad24f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 22:36:00.357055 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:36:00.357055 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:36:00.357055 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:36:00.357055 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:36:00.357055 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:36:00.357055 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:36:00.357055 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:36:00.357055 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 22:36:00.357055 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:36:00.357055 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:36:00.357055 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:36:00.357055 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:36:00.357055 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:36:00.357055 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:36:00.357055 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:36:00.357055 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:36:00.358063 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:36:00.358063 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:36:00.358063 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:36:00.358063 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:36:00.358063 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:36:00.358063 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:36:00.358063 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:36:00.358063 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:36:00.358063 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:36:00.358063 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:36:00.358063 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:36:00.358063 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:36:00.358063 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:36:00.358063 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:36:00.358063 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:36:00.358063 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:36:00.358898 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:36:00.358898 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:36:00.358898 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:36:00.358898 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:36:00.358898 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:36:00.358898 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:36:00.358898 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:36:00.358898 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:36:00.358898 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:36:00.358898 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:36:00.358898 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:36:00.358898 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:36:00.358898 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 22:36:00.358898 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:36:00.361599 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 525/1056] 10.222.0.76 () {56 vars in 1105 bytes} [Wed Aug 10 22:36:00 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:36:00.364119 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b898326-47ce-4292-81f5-8b7e149905d2 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:00.471803 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b898326-47ce-4292-81f5-8b7e149905d2 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:00.508261 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9688c-5e0c-448a-9c1b-c5dfe6fbf59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:00.508882 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9688c-5e0c-448a-9c1b-c5dfe6fbf59e None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:00.509163 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9688c-5e0c-448a-9c1b-c5dfe6fbf59e None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:00.509467 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9688c-5e0c-448a-9c1b-c5dfe6fbf59e None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:00.510149 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c9688c-5e0c-448a-9c1b-c5dfe6fbf59e None admin] RBAC: Adding query filter params (name=image) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:36:00.510455 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c9688c-5e0c-448a-9c1b-c5dfe6fbf59e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:00.511824 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c9688c-5e0c-448a-9c1b-c5dfe6fbf59e None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:00.521268 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 532/1057] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:36:00 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:36:00.522585 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2188ba-2f1b-4ae0-881e-ee5bbb4ad24f None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:00.626477 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2188ba-2f1b-4ae0-881e-ee5bbb4ad24f None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:00.660491 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4722a8b-c9a5-4f03-8af4-8b986d1ea054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:00.661243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4722a8b-c9a5-4f03-8af4-8b986d1ea054 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:00.661373 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4722a8b-c9a5-4f03-8af4-8b986d1ea054 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:00.661631 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4722a8b-c9a5-4f03-8af4-8b986d1ea054 None admin] PATH_INFO: `/v3/services` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:00.662253 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4722a8b-c9a5-4f03-8af4-8b986d1ea054 None admin] RBAC: Adding query filter params (type=image) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:36:00.662551 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4722a8b-c9a5-4f03-8af4-8b986d1ea054 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:00.663997 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4722a8b-c9a5-4f03-8af4-8b986d1ea054 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:00.665343 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:00.673909 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 526/1058] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:36:00 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:00.675782 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9688c-5e0c-448a-9c1b-c5dfe6fbf59e None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:00.738884 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:00.779491 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9688c-5e0c-448a-9c1b-c5dfe6fbf59e None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:00.812559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81a827-0f46-4c28-9ca3-b44f9b4700ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:00.813702 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81a827-0f46-4c28-9ca3-b44f9b4700ee None admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:00.813879 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81a827-0f46-4c28-9ca3-b44f9b4700ee None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:00.814117 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81a827-0f46-4c28-9ca3-b44f9b4700ee None admin] PATH_INFO: `/v3/endpoints` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:00.814772 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e81a827-0f46-4c28-9ca3-b44f9b4700ee None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:00.816131 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e81a827-0f46-4c28-9ca3-b44f9b4700ee None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:00.845303 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 533/1059] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:36:00 2022] POST /identity/v3/endpoints => generated 327 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:36:01.889422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3a58a-be7c-4402-a8f4-d77a482b8914 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:01.889952 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3a58a-be7c-4402-a8f4-d77a482b8914 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:01.889952 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3a58a-be7c-4402-a8f4-d77a482b8914 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:01.891313 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 527/1060] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:36:01.896700 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514bca99-9c95-47ad-8e81-3b286b73d0b3 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:01.897115 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514bca99-9c95-47ad-8e81-3b286b73d0b3 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:01.897306 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514bca99-9c95-47ad-8e81-3b286b73d0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:01.915750 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-514bca99-9c95-47ad-8e81-3b286b73d0b3 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:36:01.916939 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-514bca99-9c95-47ad-8e81-3b286b73d0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:36:01.996559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 534/1061] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:36:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:36:02.018389 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc3a58a-be7c-4402-a8f4-d77a482b8914 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:02.054808 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a68a4c0-80a7-45f4-b4b0-92609cb3ba38 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:02.055304 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a68a4c0-80a7-45f4-b4b0-92609cb3ba38 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:02.055650 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a68a4c0-80a7-45f4-b4b0-92609cb3ba38 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:02.056998 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 535/1062] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:36:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:36:02.062062 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27594683-b26c-404b-ad92-38e5e1c740c9 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:02.062342 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27594683-b26c-404b-ad92-38e5e1c740c9 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:02.062617 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27594683-b26c-404b-ad92-38e5e1c740c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:02.080749 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-27594683-b26c-404b-ad92-38e5e1c740c9 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:36:02.082009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-27594683-b26c-404b-ad92-38e5e1c740c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:36:02.125785 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:02.136897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc3a58a-be7c-4402-a8f4-d77a482b8914 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:02.156491 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 536/1063] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:36:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:36:02.173192 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef1021-7a44-47ca-9a6b-e3e7466151c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:02.173864 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef1021-7a44-47ca-9a6b-e3e7466151c1 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:02.174147 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef1021-7a44-47ca-9a6b-e3e7466151c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:02.174411 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef1021-7a44-47ca-9a6b-e3e7466151c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:02.180152 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27594683-b26c-404b-ad92-38e5e1c740c9 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:02.184961 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ef1021-7a44-47ca-9a6b-e3e7466151c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:36:02.189908 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-99ef1021-7a44-47ca-9a6b-e3e7466151c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:36:02.189908 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:36:02.189908 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:36:02.189908 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:36:02.189908 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:36:02.189908 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:36:02.189908 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:36:02.189908 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:36:02.189908 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:36:02.189908 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:36:02.189908 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:36:02.189908 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:36:02.189908 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:36:02.189908 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:36:02.189908 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:36:02.189908 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:36:02.190841 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:36:02.190841 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:36:02.190841 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:36:02.190841 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:36:02.190841 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:36:02.190841 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:36:02.190841 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:36:02.190841 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:36:02.190841 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:36:02.190841 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:36:02.190841 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:36:02.190841 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:36:02.190841 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:36:02.190841 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:36:02.190841 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:36:02.190841 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:36:02.191704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:36:02.191704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:36:02.191704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:36:02.191704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:36:02.191704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:36:02.191704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:36:02.191704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:36:02.191704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:36:02.191704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:36:02.191704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:36:02.191704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:36:02.191704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:36:02.191704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 528/1064] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:36:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:36:02.196992 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef1021-7a44-47ca-9a6b-e3e7466151c1 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:02.263927 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:02.294832 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27594683-b26c-404b-ad92-38e5e1c740c9 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:02.319812 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef1021-7a44-47ca-9a6b-e3e7466151c1 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:02.343476 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b0230-7812-44b5-a9fc-08fbab0f24cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:02.344118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b0230-7812-44b5-a9fc-08fbab0f24cd None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:02.344400 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b0230-7812-44b5-a9fc-08fbab0f24cd None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:02.344672 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b0230-7812-44b5-a9fc-08fbab0f24cd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:02.360343 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b0230-7812-44b5-a9fc-08fbab0f24cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:36:02.362335 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f952ae-9c33-4c53-8cf7-a920a14db34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:02.362950 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f952ae-9c33-4c53-8cf7-a920a14db34c None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:02.363226 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f952ae-9c33-4c53-8cf7-a920a14db34c None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:02.363496 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f952ae-9c33-4c53-8cf7-a920a14db34c None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:02.364152 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f952ae-9c33-4c53-8cf7-a920a14db34c None admin] RBAC: Adding query filter params (name=admin) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:36:02.364448 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f952ae-9c33-4c53-8cf7-a920a14db34c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:02.365232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-227b0230-7812-44b5-a9fc-08fbab0f24cd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 22:36:02.365232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:36:02.365232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:36:02.365232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:36:02.365232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:36:02.365232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:36:02.365232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:36:02.365232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:36:02.365232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:36:02.365232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:36:02.365232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:36:02.365232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:36:02.365232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:36:02.365232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:36:02.365232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:36:02.365232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:36:02.366288 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:36:02.366288 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:36:02.366288 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:36:02.366288 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:36:02.366288 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:36:02.366288 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:36:02.366288 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:36:02.366288 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:36:02.366288 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:36:02.366288 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:36:02.366288 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:36:02.366288 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:36:02.366288 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:36:02.366288 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:36:02.366288 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:36:02.366288 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:36:02.372607 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:36:02.372607 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:36:02.372607 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:36:02.372607 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:36:02.372607 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:36:02.372607 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:36:02.372607 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:36:02.372607 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:36:02.372607 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:36:02.372607 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:36:02.372607 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 22:36:02.372607 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:36:02.373254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f952ae-9c33-4c53-8cf7-a920a14db34c None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:02.373868 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 537/1065] 10.222.0.76 () {56 vars in 1120 bytes} [Wed Aug 10 22:36:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:36:02.373868 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b0230-7812-44b5-a9fc-08fbab0f24cd None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:02.375834 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 529/1066] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:36:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:36:02.381919 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f952ae-9c33-4c53-8cf7-a920a14db34c None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:02.422670 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:02.498663 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b0230-7812-44b5-a9fc-08fbab0f24cd None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:02.501615 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f952ae-9c33-4c53-8cf7-a920a14db34c None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:02.525672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:02.537820 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e508b7b-c215-4ecf-9205-7ccca92388cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:02.538992 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e508b7b-c215-4ecf-9205-7ccca92388cd None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:02.539398 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e508b7b-c215-4ecf-9205-7ccca92388cd None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:02.539719 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e508b7b-c215-4ecf-9205-7ccca92388cd None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:02.540394 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e508b7b-c215-4ecf-9205-7ccca92388cd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:36:02.540699 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e508b7b-c215-4ecf-9205-7ccca92388cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:02.541645 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7a8db8-def4-4c81-a445-f9a0a2a00cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:02.542287 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a8db8-def4-4c81-a445-f9a0a2a00cdb None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:02.542512 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e508b7b-c215-4ecf-9205-7ccca92388cd None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:02.542619 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a8db8-def4-4c81-a445-f9a0a2a00cdb None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:02.542823 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a8db8-def4-4c81-a445-f9a0a2a00cdb None admin] PATH_INFO: `/v3/users/6b051279581945c69892101879b2ad4e` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:02.543965 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7a8db8-def4-4c81-a445-f9a0a2a00cdb None admin] RBAC: Authorizing `identity:get_user(user_id=6b051279581945c69892101879b2ad4e)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:02.545428 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7a8db8-def4-4c81-a445-f9a0a2a00cdb None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:02.547242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 530/1067] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:36:02 2022] GET /identity/v3/users/6b051279581945c69892101879b2ad4e => generated 285 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:02.552742 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7a8db8-def4-4c81-a445-f9a0a2a00cdb None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:02.554286 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 538/1068] 10.222.0.76 () {56 vars in 1115 bytes} [Wed Aug 10 22:36:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:02.560187 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e508b7b-c215-4ecf-9205-7ccca92388cd None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:02.674419 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7a8db8-def4-4c81-a445-f9a0a2a00cdb None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:02.682824 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:02.691353 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e508b7b-c215-4ecf-9205-7ccca92388cd None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:02.718507 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a4010-9c45-425a-be0b-279b0b4ae954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:02.719303 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a4010-9c45-425a-be0b-279b0b4ae954 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:02.719417 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a4010-9c45-425a-be0b-279b0b4ae954 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:02.719691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a4010-9c45-425a-be0b-279b0b4ae954 None admin] PATH_INFO: `/v3/projects/9e62fda33d2843ca9cd1614d910b1fb9` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:02.720839 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051a4010-9c45-425a-be0b-279b0b4ae954 None admin] RBAC: Authorizing `identity:get_project(project_id=9e62fda33d2843ca9cd1614d910b1fb9)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:02.722320 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051a4010-9c45-425a-be0b-279b0b4ae954 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:02.724146 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 531/1069] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:36:02 2022] GET /identity/v3/projects/9e62fda33d2843ca9cd1614d910b1fb9 => generated 311 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:36:02.730994 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a4010-9c45-425a-be0b-279b0b4ae954 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:02.731168 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3543e3c-d296-47b1-a420-c3eae635ee4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:02.731834 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3543e3c-d296-47b1-a420-c3eae635ee4e None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:02.732122 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3543e3c-d296-47b1-a420-c3eae635ee4e None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:02.732397 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3543e3c-d296-47b1-a420-c3eae635ee4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:02.744653 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3543e3c-d296-47b1-a420-c3eae635ee4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:02.746147 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3543e3c-d296-47b1-a420-c3eae635ee4e None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:02.764486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-b3543e3c-d296-47b1-a420-c3eae635ee4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:36:02.764486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:36:02.764486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:36:02.764486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:36:02.764486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:36:02.764486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:36:02.764486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:36:02.764486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:36:02.764486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:36:02.764486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:36:02.764486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:36:02.764486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:36:02.764486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:36:02.764486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:36:02.764486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:36:02.764486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:36:02.765627 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:36:02.765627 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:36:02.765627 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:36:02.765627 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:36:02.765627 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:36:02.765627 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:36:02.765627 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:36:02.765627 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:36:02.765627 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:36:02.765627 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:36:02.765627 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:36:02.765627 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:36:02.765627 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:36:02.765627 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:36:02.765627 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:36:02.765627 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:36:02.765627 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:36:02.766527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:36:02.766527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:36:02.766527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:36:02.766527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 539/1070] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:36:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:36:02.771333 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3543e3c-d296-47b1-a420-c3eae635ee4e None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:02.800601 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:02.848753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a4010-9c45-425a-be0b-279b0b4ae954 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:02.885239 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d5f63-fc4c-42b4-9fed-3a893b3e2c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:02.886287 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d5f63-fc4c-42b4-9fed-3a893b3e2c99 None admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:02.886783 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d5f63-fc4c-42b4-9fed-3a893b3e2c99 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:02.887265 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d5f63-fc4c-42b4-9fed-3a893b3e2c99 None admin] PATH_INFO: `/v3/projects/9e62fda33d2843ca9cd1614d910b1fb9/users/6b051279581945c69892101879b2ad4e/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:02.889550 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d5f63-fc4c-42b4-9fed-3a893b3e2c99 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e62fda33d2843ca9cd1614d910b1fb9, user_id=6b051279581945c69892101879b2ad4e, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:02.891658 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d5f63-fc4c-42b4-9fed-3a893b3e2c99 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:02.907177 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 532/1071] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 10 22:36:02 2022] PUT /identity/v3/projects/9e62fda33d2843ca9cd1614d910b1fb9/users/6b051279581945c69892101879b2ad4e/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:36:02.908537 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11fac2b-adab-4de9-8381-3fc23aee7ce1 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:02.908822 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11fac2b-adab-4de9-8381-3fc23aee7ce1 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:02.909090 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11fac2b-adab-4de9-8381-3fc23aee7ce1 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:02.910492 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 533/1072] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:36:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:36:02.911303 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3543e3c-d296-47b1-a420-c3eae635ee4e None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:02.916839 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20629e-7a0e-4c6c-836a-9565f1474db2 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:02.917168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20629e-7a0e-4c6c-836a-9565f1474db2 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:02.917425 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20629e-7a0e-4c6c-836a-9565f1474db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:02.938366 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-6c20629e-7a0e-4c6c-836a-9565f1474db2 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:36:02.939504 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-6c20629e-7a0e-4c6c-836a-9565f1474db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:36:02.953205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f891d3c4-75a1-4fbb-9b7d-b43ff9541b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:02.953939 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891d3c4-75a1-4fbb-9b7d-b43ff9541b11 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:02.954229 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891d3c4-75a1-4fbb-9b7d-b43ff9541b11 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:02.954505 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891d3c4-75a1-4fbb-9b7d-b43ff9541b11 None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:02.955167 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f891d3c4-75a1-4fbb-9b7d-b43ff9541b11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:36:02.955475 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f891d3c4-75a1-4fbb-9b7d-b43ff9541b11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:02.956896 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f891d3c4-75a1-4fbb-9b7d-b43ff9541b11 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:02.977695 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 540/1073] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:36:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:36:02.979355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f891d3c4-75a1-4fbb-9b7d-b43ff9541b11 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:02.991517 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:03.027508 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 534/1074] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:36:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:36:03.050916 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20629e-7a0e-4c6c-836a-9565f1474db2 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:03.098606 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f891d3c4-75a1-4fbb-9b7d-b43ff9541b11 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:03.112987 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:03.135708 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58682dfc-c9ca-49fc-a110-5a8f612d4bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:03.136347 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58682dfc-c9ca-49fc-a110-5a8f612d4bd9 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:03.136623 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58682dfc-c9ca-49fc-a110-5a8f612d4bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:03.136894 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58682dfc-c9ca-49fc-a110-5a8f612d4bd9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:03.144430 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58682dfc-c9ca-49fc-a110-5a8f612d4bd9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:03.146071 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58682dfc-c9ca-49fc-a110-5a8f612d4bd9 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:03.158310 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-58682dfc-c9ca-49fc-a110-5a8f612d4bd9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:36:03.158310 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:36:03.158310 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:36:03.158310 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:36:03.158310 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:36:03.158310 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:36:03.158310 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:36:03.158310 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:36:03.158310 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:36:03.158310 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:36:03.158310 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:36:03.158310 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:36:03.158310 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:36:03.158310 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:36:03.158310 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:36:03.158310 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:36:03.159530 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:36:03.159530 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:36:03.159530 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:36:03.159530 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:36:03.159530 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:36:03.159530 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:36:03.159530 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:36:03.159530 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:36:03.159530 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:36:03.159530 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:36:03.159530 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:36:03.159530 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:36:03.159530 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:36:03.159530 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:36:03.159530 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:36:03.159530 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:36:03.159530 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:36:03.159530 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:36:03.160508 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:36:03.160508 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:36:03.160508 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:36:03.160508 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:36:03.160508 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:36:03.160508 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:36:03.160508 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:36:03.160508 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:36:03.160508 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:36:03.160508 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:36:03.160508 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 541/1075] 10.222.0.76 () {56 vars in 1093 bytes} [Wed Aug 10 22:36:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:36:03.166352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58682dfc-c9ca-49fc-a110-5a8f612d4bd9 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:03.180074 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20629e-7a0e-4c6c-836a-9565f1474db2 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:03.220226 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a3bdf-6526-4fcf-a535-3b475487aa63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:03.220757 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a3bdf-6526-4fcf-a535-3b475487aa63 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:03.221034 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a3bdf-6526-4fcf-a535-3b475487aa63 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:03.221329 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a3bdf-6526-4fcf-a535-3b475487aa63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:03.239880 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:03.278705 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58682dfc-c9ca-49fc-a110-5a8f612d4bd9 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:03.297011 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a3bdf-6526-4fcf-a535-3b475487aa63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:03.298598 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a3bdf-6526-4fcf-a535-3b475487aa63 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:03.316485 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ac597-855a-4d24-8550-cdebc9276c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:03.317241 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ac597-855a-4d24-8550-cdebc9276c51 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:03.317552 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ac597-855a-4d24-8550-cdebc9276c51 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:03.317824 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ac597-855a-4d24-8550-cdebc9276c51 None admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:03.318546 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92ac597-855a-4d24-8550-cdebc9276c51 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:36:03.318848 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92ac597-855a-4d24-8550-cdebc9276c51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:03.320416 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92ac597-855a-4d24-8550-cdebc9276c51 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:03.338861 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 542/1076] 10.222.0.76 () {56 vars in 1133 bytes} [Wed Aug 10 22:36:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:36:03.344515 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ac597-855a-4d24-8550-cdebc9276c51 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:03.352991 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:03.412655 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 535/1077] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:36:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 22:36:03.421750 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a3bdf-6526-4fcf-a535-3b475487aa63 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:03.472580 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ac597-855a-4d24-8550-cdebc9276c51 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:03.500190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:03.510553 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4af9d0-a524-4010-9dac-aceaebb67106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:03.511280 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4af9d0-a524-4010-9dac-aceaebb67106 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:03.511631 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4af9d0-a524-4010-9dac-aceaebb67106 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:03.512244 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4af9d0-a524-4010-9dac-aceaebb67106 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:03.523532 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4af9d0-a524-4010-9dac-aceaebb67106 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:03.525069 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4af9d0-a524-4010-9dac-aceaebb67106 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:03.532376 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a3bdf-6526-4fcf-a535-3b475487aa63 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:03.540835 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-ef4af9d0-a524-4010-9dac-aceaebb67106 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:36:03.540835 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:36:03.540835 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:36:03.540835 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:36:03.540835 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:36:03.540835 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:36:03.540835 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:36:03.540835 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:36:03.540835 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:36:03.540835 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:36:03.540835 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:36:03.540835 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:36:03.540835 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:36:03.540835 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:36:03.540835 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:36:03.540835 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:36:03.541827 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:36:03.541827 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:36:03.541827 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:36:03.541827 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:36:03.541827 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:36:03.541827 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:36:03.541827 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:36:03.541827 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:36:03.541827 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:36:03.541827 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:36:03.541827 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:36:03.541827 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:36:03.541827 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:36:03.541827 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:36:03.541827 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:36:03.541827 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:36:03.541827 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:36:03.542731 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:36:03.542731 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:36:03.542731 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:36:03.542731 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 543/1078] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:36:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:36:03.547533 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4af9d0-a524-4010-9dac-aceaebb67106 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:03.574335 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128e9e5-76e8-488e-a9ae-dd6c03b9f66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:03.574967 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128e9e5-76e8-488e-a9ae-dd6c03b9f66f None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:03.575244 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128e9e5-76e8-488e-a9ae-dd6c03b9f66f None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:03.575514 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128e9e5-76e8-488e-a9ae-dd6c03b9f66f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:03.587187 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7128e9e5-76e8-488e-a9ae-dd6c03b9f66f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:03.588642 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7128e9e5-76e8-488e-a9ae-dd6c03b9f66f None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:03.596583 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:03.605700 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-7128e9e5-76e8-488e-a9ae-dd6c03b9f66f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:36:03.605700 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:36:03.605700 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:36:03.605700 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:36:03.605700 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:36:03.605700 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:36:03.605700 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:36:03.605700 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:36:03.605700 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:36:03.605700 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:36:03.605700 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:36:03.605700 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:36:03.605700 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:36:03.605700 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:36:03.605700 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:36:03.605700 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:36:03.606730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:36:03.606730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:36:03.606730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:36:03.606730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:36:03.606730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:36:03.606730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:36:03.606730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:36:03.606730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:36:03.606730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:36:03.606730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:36:03.606730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:36:03.606730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:36:03.606730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:36:03.606730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:36:03.606730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:36:03.606730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:36:03.606730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:36:03.607656 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:36:03.607656 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:36:03.607656 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:36:03.607656 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 536/1079] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:36:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:36:03.612651 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128e9e5-76e8-488e-a9ae-dd6c03b9f66f None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:03.657893 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4af9d0-a524-4010-9dac-aceaebb67106 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:03.693869 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1fb98-b4c6-4223-9c8d-288ddd0012ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:03.695160 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1fb98-b4c6-4223-9c8d-288ddd0012ab None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:03.695448 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1fb98-b4c6-4223-9c8d-288ddd0012ab None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:03.695724 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1fb98-b4c6-4223-9c8d-288ddd0012ab None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:03.696369 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef1fb98-b4c6-4223-9c8d-288ddd0012ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:36:03.696670 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef1fb98-b4c6-4223-9c8d-288ddd0012ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:03.698093 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef1fb98-b4c6-4223-9c8d-288ddd0012ab None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:03.710402 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 544/1080] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:36:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:36:03.715639 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1fb98-b4c6-4223-9c8d-288ddd0012ab None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:03.727685 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128e9e5-76e8-488e-a9ae-dd6c03b9f66f None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:03.759528 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:03.765807 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f2091-3616-41f7-ae36-8feec982409e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:03.766420 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f2091-3616-41f7-ae36-8feec982409e None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:03.766698 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f2091-3616-41f7-ae36-8feec982409e None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:03.766967 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f2091-3616-41f7-ae36-8feec982409e None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:03.767595 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f2091-3616-41f7-ae36-8feec982409e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:36:03.767896 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f2091-3616-41f7-ae36-8feec982409e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:03.769365 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f2091-3616-41f7-ae36-8feec982409e None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:03.781537 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 537/1081] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:36:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:36:03.826900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1fb98-b4c6-4223-9c8d-288ddd0012ab None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:03.864688 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c568d4-e7ed-40e6-bf35-87c7e17b6ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:03.865392 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c568d4-e7ed-40e6-bf35-87c7e17b6ba5 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:03.865678 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c568d4-e7ed-40e6-bf35-87c7e17b6ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:03.865954 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c568d4-e7ed-40e6-bf35-87c7e17b6ba5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:03.874462 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c568d4-e7ed-40e6-bf35-87c7e17b6ba5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:03.875947 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c568d4-e7ed-40e6-bf35-87c7e17b6ba5 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:03.888733 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-16c568d4-e7ed-40e6-bf35-87c7e17b6ba5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:36:03.888733 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:36:03.888733 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:36:03.888733 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:36:03.888733 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:36:03.888733 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:36:03.888733 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:36:03.888733 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:36:03.888733 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:36:03.888733 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:36:03.888733 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:36:03.888733 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:36:03.888733 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:36:03.888733 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:36:03.888733 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:36:03.889776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:36:03.889776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:36:03.889776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:36:03.889776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:36:03.889776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:36:03.889776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:36:03.889776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:36:03.889776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:36:03.889776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:36:03.889776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:36:03.889776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:36:03.889776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:36:03.889776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:36:03.889776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:36:03.889776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:36:03.889776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:36:03.889776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:36:03.890696 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:36:03.890696 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:36:03.890696 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:36:03.890696 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:36:03.890696 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:36:03.890696 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:36:03.890696 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:36:03.890696 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:36:03.893645 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 545/1082] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:36:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:36:03.895832 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f2091-3616-41f7-ae36-8feec982409e None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:03.960506 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:04.006706 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f2091-3616-41f7-ae36-8feec982409e None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:04.045468 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba0d63-3fd1-4aea-9b2b-1baa1419a904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:04.046088 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba0d63-3fd1-4aea-9b2b-1baa1419a904 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:04.046365 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba0d63-3fd1-4aea-9b2b-1baa1419a904 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:04.046633 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba0d63-3fd1-4aea-9b2b-1baa1419a904 None admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:04.047325 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dba0d63-3fd1-4aea-9b2b-1baa1419a904 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:36:04.047623 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dba0d63-3fd1-4aea-9b2b-1baa1419a904 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:04.049038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dba0d63-3fd1-4aea-9b2b-1baa1419a904 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:04.063503 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 538/1083] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:36:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:04.069168 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c568d4-e7ed-40e6-bf35-87c7e17b6ba5 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:04.167852 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:04.171875 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c568d4-e7ed-40e6-bf35-87c7e17b6ba5 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:04.205486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d5873-8a3a-4876-bf26-410db1e0cd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:04.206132 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d5873-8a3a-4876-bf26-410db1e0cd95 None admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:04.206414 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d5873-8a3a-4876-bf26-410db1e0cd95 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:04.206683 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d5873-8a3a-4876-bf26-410db1e0cd95 None admin] PATH_INFO: `/v3/projects/3caa2ea846f7449b8c483788fc5846c8/users/b22838a8ad3949e59cdbaaa1801cd07f/roles/ee973ce49dc142a6b0763794e078333d` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:04.208773 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d5873-8a3a-4876-bf26-410db1e0cd95 None admin] RBAC: Authorizing `identity:create_grant(project_id=3caa2ea846f7449b8c483788fc5846c8, user_id=b22838a8ad3949e59cdbaaa1801cd07f, role_id=ee973ce49dc142a6b0763794e078333d)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:04.210372 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d5873-8a3a-4876-bf26-410db1e0cd95 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:04.222643 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 546/1084] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 10 22:36:04 2022] PUT /identity/v3/projects/3caa2ea846f7449b8c483788fc5846c8/users/b22838a8ad3949e59cdbaaa1801cd07f/roles/ee973ce49dc142a6b0763794e078333d => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:36:05.009199 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff9fe5-6eea-411b-9d95-ac3de28e2706 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:05.009761 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff9fe5-6eea-411b-9d95-ac3de28e2706 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:05.009902 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff9fe5-6eea-411b-9d95-ac3de28e2706 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:05.011264 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 539/1085] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:36:05.016591 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1d1b0-5076-4076-b7c7-ce5d722ebf6d None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:05.016903 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1d1b0-5076-4076-b7c7-ce5d722ebf6d None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:05.017163 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1d1b0-5076-4076-b7c7-ce5d722ebf6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:05.035734 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-23a1d1b0-5076-4076-b7c7-ce5d722ebf6d None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:36:05.036917 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-23a1d1b0-5076-4076-b7c7-ce5d722ebf6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:36:05.103523 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 547/1086] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:36:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:36:05.126523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff9fe5-6eea-411b-9d95-ac3de28e2706 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:05.230335 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff9fe5-6eea-411b-9d95-ac3de28e2706 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:05.247699 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:05.266076 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ba732-80f5-477e-96d8-4edc7657f51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:05.266642 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ba732-80f5-477e-96d8-4edc7657f51f None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:05.266923 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ba732-80f5-477e-96d8-4edc7657f51f None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:05.267195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ba732-80f5-477e-96d8-4edc7657f51f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:05.278082 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81ba732-80f5-477e-96d8-4edc7657f51f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:36:05.282726 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-e81ba732-80f5-477e-96d8-4edc7657f51f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:36:05.282726 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:36:05.282726 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:36:05.282726 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:36:05.282726 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:36:05.282726 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:36:05.282726 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:36:05.282726 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:36:05.282726 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:36:05.282726 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:36:05.282726 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:36:05.282726 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:36:05.282726 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:36:05.282726 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:36:05.282726 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:36:05.282726 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:36:05.283754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:36:05.283754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:36:05.283754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:36:05.283754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:36:05.283754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:36:05.283754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:36:05.283754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:36:05.283754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:36:05.283754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:36:05.283754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:36:05.283754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:36:05.283754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:36:05.283754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:36:05.283754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:36:05.283754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:36:05.283754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:36:05.284639 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:36:05.284639 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:36:05.284639 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:36:05.284639 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:36:05.284639 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:36:05.284639 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:36:05.284639 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:36:05.284639 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:36:05.284639 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:36:05.284639 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:36:05.284639 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:36:05.284639 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:36:05.284639 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 540/1087] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:36:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:36:05.289905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1d1b0-5076-4076-b7c7-ce5d722ebf6d None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:05.378429 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:05.392297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1d1b0-5076-4076-b7c7-ce5d722ebf6d None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:05.434405 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ca559-e48c-4fe6-9b92-092d0f2e1635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:05.435031 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ca559-e48c-4fe6-9b92-092d0f2e1635 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:05.435316 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ca559-e48c-4fe6-9b92-092d0f2e1635 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:05.435594 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ca559-e48c-4fe6-9b92-092d0f2e1635 None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:05.436264 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ca559-e48c-4fe6-9b92-092d0f2e1635 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:36:05.436569 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ca559-e48c-4fe6-9b92-092d0f2e1635 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:05.438092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ca559-e48c-4fe6-9b92-092d0f2e1635 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:05.447850 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 548/1088] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:36:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:05.453684 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ba732-80f5-477e-96d8-4edc7657f51f None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:05.555486 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ba732-80f5-477e-96d8-4edc7657f51f None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:05.589335 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c28b5-b582-44f7-8348-3d7109d73134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:05.589962 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c28b5-b582-44f7-8348-3d7109d73134 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:05.590239 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c28b5-b582-44f7-8348-3d7109d73134 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:05.590510 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c28b5-b582-44f7-8348-3d7109d73134 None admin] PATH_INFO: `/v3/users/6b051279581945c69892101879b2ad4e` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:05.591628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c28b5-b582-44f7-8348-3d7109d73134 None admin] RBAC: Authorizing `identity:get_user(user_id=6b051279581945c69892101879b2ad4e)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:05.593051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c28b5-b582-44f7-8348-3d7109d73134 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:05.594663 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 541/1089] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:36:05 2022] GET /identity/v3/users/6b051279581945c69892101879b2ad4e => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:05.600481 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ca559-e48c-4fe6-9b92-092d0f2e1635 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:05.710149 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ca559-e48c-4fe6-9b92-092d0f2e1635 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:05.742579 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a740792-49cd-4be2-a9cc-047b3e09b25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:05.743175 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a740792-49cd-4be2-a9cc-047b3e09b25d None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:05.743453 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a740792-49cd-4be2-a9cc-047b3e09b25d None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:05.743721 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a740792-49cd-4be2-a9cc-047b3e09b25d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:05.762343 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:05.810434 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a740792-49cd-4be2-a9cc-047b3e09b25d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:05.811800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a740792-49cd-4be2-a9cc-047b3e09b25d None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:05.913756 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 549/1090] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:36:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 22:36:05.919606 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c28b5-b582-44f7-8348-3d7109d73134 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:05.967040 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:06.019339 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c28b5-b582-44f7-8348-3d7109d73134 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:06.055749 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb18286-2ed4-4a1d-8e5b-9603639f6b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:06.056371 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb18286-2ed4-4a1d-8e5b-9603639f6b72 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:06.056646 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb18286-2ed4-4a1d-8e5b-9603639f6b72 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:06.056912 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb18286-2ed4-4a1d-8e5b-9603639f6b72 None admin] PATH_INFO: `/v3/projects/9e62fda33d2843ca9cd1614d910b1fb9` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:06.058048 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb18286-2ed4-4a1d-8e5b-9603639f6b72 None admin] RBAC: Authorizing `identity:get_project(project_id=9e62fda33d2843ca9cd1614d910b1fb9)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:06.059459 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb18286-2ed4-4a1d-8e5b-9603639f6b72 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:06.061142 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 542/1091] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:36:05 2022] GET /identity/v3/projects/9e62fda33d2843ca9cd1614d910b1fb9 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:36:06.067512 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a740792-49cd-4be2-a9cc-047b3e09b25d None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:06.119035 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:06.169152 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a740792-49cd-4be2-a9cc-047b3e09b25d None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:06.204359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc4753-6885-40d5-abff-603934ad0f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:06.204964 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc4753-6885-40d5-abff-603934ad0f72 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:06.205245 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc4753-6885-40d5-abff-603934ad0f72 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:06.205544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc4753-6885-40d5-abff-603934ad0f72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:06.206266 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fc4753-6885-40d5-abff-603934ad0f72 None admin] RBAC: Adding query filter params (role.id=6641e95fdc1f4c098ebc372bb692d227, scope.project.id=9e62fda33d2843ca9cd1614d910b1fb9, user.id=6b051279581945c69892101879b2ad4e) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:36:06.206571 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fc4753-6885-40d5-abff-603934ad0f72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:06.207997 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fc4753-6885-40d5-abff-603934ad0f72 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:06.214381 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 550/1092] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:36:06 2022] GET /identity/v3/role_assignments?user.id=6b051279581945c69892101879b2ad4e&scope.project.id=9e62fda33d2843ca9cd1614d910b1fb9&role.id=6641e95fdc1f4c098ebc372bb692d227 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:06.248574 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d690fa-2fda-471b-8f49-55d0bd5eb053 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:06.248865 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d690fa-2fda-471b-8f49-55d0bd5eb053 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:06.249146 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d690fa-2fda-471b-8f49-55d0bd5eb053 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:06.250575 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 543/1093] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:36:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:36:06.256556 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbebd9de-715f-4e9f-8ddf-4aba119e001a None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:06.256841 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbebd9de-715f-4e9f-8ddf-4aba119e001a None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:06.257687 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbebd9de-715f-4e9f-8ddf-4aba119e001a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:06.277217 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-cbebd9de-715f-4e9f-8ddf-4aba119e001a None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:36:06.278475 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-cbebd9de-715f-4e9f-8ddf-4aba119e001a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:36:06.349218 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 551/1094] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:36:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:36:06.375026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d690fa-2fda-471b-8f49-55d0bd5eb053 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:06.483635 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d690fa-2fda-471b-8f49-55d0bd5eb053 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:06.497364 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:06.517792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0491706-510b-4cff-9806-3cb63e61b38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:06.518418 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0491706-510b-4cff-9806-3cb63e61b38d None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:06.518692 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0491706-510b-4cff-9806-3cb63e61b38d None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:06.518962 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0491706-510b-4cff-9806-3cb63e61b38d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:06.530012 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0491706-510b-4cff-9806-3cb63e61b38d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:36:06.534572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-e0491706-510b-4cff-9806-3cb63e61b38d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 22:36:06.534572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:36:06.534572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:36:06.534572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:36:06.534572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:36:06.534572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:36:06.534572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:36:06.534572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:36:06.534572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:36:06.534572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:36:06.534572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:36:06.534572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:36:06.534572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:36:06.534572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:36:06.534572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:36:06.534572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:36:06.535590 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:36:06.535590 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:36:06.535590 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:36:06.535590 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:36:06.535590 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:36:06.535590 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:36:06.535590 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:36:06.535590 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:36:06.535590 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:36:06.535590 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:36:06.535590 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:36:06.535590 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:36:06.535590 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:36:06.535590 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:36:06.535590 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:36:06.535590 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:36:06.536462 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:36:06.536462 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:36:06.536462 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:36:06.536462 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:36:06.536462 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:36:06.536462 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:36:06.536462 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:36:06.536462 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:36:06.536462 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:36:06.536462 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:36:06.536462 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 22:36:06.536462 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:36:06.536462 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 544/1095] 10.222.0.76 () {56 vars in 1120 bytes} [Wed Aug 10 22:36:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:36:06.541852 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbebd9de-715f-4e9f-8ddf-4aba119e001a None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:06.588882 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:06.643032 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbebd9de-715f-4e9f-8ddf-4aba119e001a None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:06.676507 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbedaf2-aaa5-4c41-bf37-8c82554c6158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:06.677014 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbedaf2-aaa5-4c41-bf37-8c82554c6158 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:06.677316 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbedaf2-aaa5-4c41-bf37-8c82554c6158 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:06.677595 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbedaf2-aaa5-4c41-bf37-8c82554c6158 None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:06.678250 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbedaf2-aaa5-4c41-bf37-8c82554c6158 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:36:06.678550 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbedaf2-aaa5-4c41-bf37-8c82554c6158 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:06.680469 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbedaf2-aaa5-4c41-bf37-8c82554c6158 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:06.688970 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 552/1096] 10.222.0.76 () {56 vars in 1115 bytes} [Wed Aug 10 22:36:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:06.695191 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0491706-510b-4cff-9806-3cb63e61b38d None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:06.794771 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0491706-510b-4cff-9806-3cb63e61b38d None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:06.828997 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d28d3-5fa2-409a-9046-50cb17946f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:06.829656 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d28d3-5fa2-409a-9046-50cb17946f1d None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:06.829940 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d28d3-5fa2-409a-9046-50cb17946f1d None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:06.830211 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d28d3-5fa2-409a-9046-50cb17946f1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:06.841603 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d28d3-5fa2-409a-9046-50cb17946f1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:06.843046 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d28d3-5fa2-409a-9046-50cb17946f1d None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:06.857157 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-f03d28d3-5fa2-409a-9046-50cb17946f1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:36:06.857157 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:36:06.857157 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:36:06.857157 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:36:06.857157 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:36:06.857157 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:36:06.857157 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:36:06.857157 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:36:06.857157 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:36:06.857157 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:36:06.857157 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:36:06.857157 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:36:06.857157 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:36:06.857157 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:36:06.857157 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:36:06.857157 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:36:06.858302 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:36:06.858302 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:36:06.858302 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:36:06.858302 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:36:06.858302 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:36:06.858302 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:36:06.858302 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:36:06.858302 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:36:06.858302 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:36:06.858302 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:36:06.858302 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:36:06.858302 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:36:06.858302 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:36:06.858302 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:36:06.858302 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:36:06.858302 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:36:06.858302 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:36:06.859216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:36:06.859216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:36:06.859216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:36:06.859216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 545/1097] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:36:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:36:06.864015 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbedaf2-aaa5-4c41-bf37-8c82554c6158 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:06.964712 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbedaf2-aaa5-4c41-bf37-8c82554c6158 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:06.971530 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:07.000046 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a49b7-d5ae-4c5c-95cf-b2838b6dc006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:07.000653 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a49b7-d5ae-4c5c-95cf-b2838b6dc006 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:07.000930 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a49b7-d5ae-4c5c-95cf-b2838b6dc006 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:07.001198 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a49b7-d5ae-4c5c-95cf-b2838b6dc006 None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:07.001856 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a49b7-d5ae-4c5c-95cf-b2838b6dc006 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:36:07.002153 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a49b7-d5ae-4c5c-95cf-b2838b6dc006 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:07.003509 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a49b7-d5ae-4c5c-95cf-b2838b6dc006 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:07.015282 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 553/1098] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:36:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:07.021304 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d28d3-5fa2-409a-9046-50cb17946f1d None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:07.049585 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:07.122236 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d28d3-5fa2-409a-9046-50cb17946f1d None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:07.155241 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb997a8f-6483-4a13-bbcf-cc5541ed4569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:07.155856 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb997a8f-6483-4a13-bbcf-cc5541ed4569 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:07.156129 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb997a8f-6483-4a13-bbcf-cc5541ed4569 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:07.156395 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb997a8f-6483-4a13-bbcf-cc5541ed4569 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:07.163620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb997a8f-6483-4a13-bbcf-cc5541ed4569 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:07.165057 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb997a8f-6483-4a13-bbcf-cc5541ed4569 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:07.175628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-eb997a8f-6483-4a13-bbcf-cc5541ed4569 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:36:07.175628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:36:07.175628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:36:07.175628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:36:07.175628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:36:07.175628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:36:07.175628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:36:07.175628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:36:07.175628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:36:07.175628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:36:07.175628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:36:07.175628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:36:07.175628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:36:07.175628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:36:07.175628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:36:07.175628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:36:07.176621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:36:07.176621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:36:07.176621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:36:07.176621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:36:07.176621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:36:07.176621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:36:07.176621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:36:07.176621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:36:07.176621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:36:07.176621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:36:07.176621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:36:07.176621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:36:07.176621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:36:07.176621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:36:07.176621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:36:07.176621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:36:07.176621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:36:07.176621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:36:07.177731 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:36:07.177731 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:36:07.177731 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:36:07.177731 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:36:07.177731 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:36:07.177731 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:36:07.177731 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:36:07.177731 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:36:07.177731 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 22:36:07.177731 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:36:07.177731 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 546/1099] 10.222.0.76 () {56 vars in 1093 bytes} [Wed Aug 10 22:36:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:36:07.182434 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a49b7-d5ae-4c5c-95cf-b2838b6dc006 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:07.294551 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a49b7-d5ae-4c5c-95cf-b2838b6dc006 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:07.328925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc4d9c5-ed61-44af-a1ae-e80610135106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:07.329567 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4d9c5-ed61-44af-a1ae-e80610135106 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:07.329848 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4d9c5-ed61-44af-a1ae-e80610135106 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:07.330120 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4d9c5-ed61-44af-a1ae-e80610135106 None admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:07.330849 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc4d9c5-ed61-44af-a1ae-e80610135106 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:36:07.331153 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc4d9c5-ed61-44af-a1ae-e80610135106 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:07.332574 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc4d9c5-ed61-44af-a1ae-e80610135106 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:07.350485 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 554/1100] 10.222.0.76 () {56 vars in 1133 bytes} [Wed Aug 10 22:36:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:07.357267 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb997a8f-6483-4a13-bbcf-cc5541ed4569 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:07.450940 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:07.466977 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb997a8f-6483-4a13-bbcf-cc5541ed4569 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:07.500187 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99058a1-1632-4e89-ac13-b6c14392583e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:07.500791 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99058a1-1632-4e89-ac13-b6c14392583e None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:07.501071 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99058a1-1632-4e89-ac13-b6c14392583e None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:07.501372 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99058a1-1632-4e89-ac13-b6c14392583e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:07.570425 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99058a1-1632-4e89-ac13-b6c14392583e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:07.571981 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99058a1-1632-4e89-ac13-b6c14392583e None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:07.660625 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:07.680431 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 547/1101] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:36:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 22:36:07.686238 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc4d9c5-ed61-44af-a1ae-e80610135106 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:07.712126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:07.785079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc4d9c5-ed61-44af-a1ae-e80610135106 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:07.818682 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa1438f-dd87-492c-b7be-fe2dcf81f55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:07.819307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1438f-dd87-492c-b7be-fe2dcf81f55a None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:07.820121 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1438f-dd87-492c-b7be-fe2dcf81f55a None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:07.820392 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1438f-dd87-492c-b7be-fe2dcf81f55a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:07.831684 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa1438f-dd87-492c-b7be-fe2dcf81f55a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:07.831824 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa1438f-dd87-492c-b7be-fe2dcf81f55a None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:07.846790 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-faa1438f-dd87-492c-b7be-fe2dcf81f55a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:36:07.846790 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:36:07.846790 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:36:07.846790 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:36:07.846790 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:36:07.846790 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:36:07.846790 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:36:07.846790 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:36:07.846790 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:36:07.846790 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:36:07.846790 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:36:07.846790 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:36:07.846790 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:36:07.846790 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:36:07.846790 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:36:07.846790 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:36:07.847937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:36:07.847937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:36:07.847937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:36:07.847937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:36:07.847937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:36:07.847937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:36:07.847937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:36:07.847937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:36:07.847937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:36:07.847937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:36:07.847937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:36:07.847937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:36:07.847937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:36:07.847937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:36:07.847937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:36:07.847937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:36:07.847937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:36:07.849378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:36:07.849378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:36:07.849378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:36:07.849378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 555/1102] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:36:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:36:07.854330 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99058a1-1632-4e89-ac13-b6c14392583e None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:07.951781 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99058a1-1632-4e89-ac13-b6c14392583e None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:07.985987 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faf67b8-5531-476d-9380-459979c98314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:07.986627 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf67b8-5531-476d-9380-459979c98314 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:07.987017 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf67b8-5531-476d-9380-459979c98314 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:07.987372 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf67b8-5531-476d-9380-459979c98314 None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:07.988250 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faf67b8-5531-476d-9380-459979c98314 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:36:07.988853 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faf67b8-5531-476d-9380-459979c98314 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:07.989938 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faf67b8-5531-476d-9380-459979c98314 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:08.001830 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 548/1103] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:36:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:08.007591 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa1438f-dd87-492c-b7be-fe2dcf81f55a None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:08.088043 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:08.111388 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa1438f-dd87-492c-b7be-fe2dcf81f55a None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:08.146142 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fec9828-3689-4268-8447-ab10d5b3670c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:08.146774 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec9828-3689-4268-8447-ab10d5b3670c None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:08.147078 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec9828-3689-4268-8447-ab10d5b3670c None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:08.147344 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec9828-3689-4268-8447-ab10d5b3670c None admin] PATH_INFO: `/v3/projects/service` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:08.154192 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fec9828-3689-4268-8447-ab10d5b3670c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:08.155641 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fec9828-3689-4268-8447-ab10d5b3670c None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:08.166690 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-4fec9828-3689-4268-8447-ab10d5b3670c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:36:08.166690 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:36:08.166690 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:36:08.166690 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:36:08.166690 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:36:08.166690 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:36:08.166690 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:36:08.166690 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:36:08.166690 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:36:08.166690 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:36:08.166690 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:36:08.166690 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:36:08.166690 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:36:08.166690 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:36:08.166690 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:36:08.167787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:36:08.167787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:36:08.167787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:36:08.167787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:36:08.167787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:36:08.167787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:36:08.167787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:36:08.167787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:36:08.167787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:36:08.167787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:36:08.167787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:36:08.167787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:36:08.167787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:36:08.167787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:36:08.167787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:36:08.167787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:36:08.167787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:36:08.168738 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:36:08.168738 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:36:08.168738 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:36:08.168738 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:36:08.168738 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:36:08.168738 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:36:08.168738 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:36:08.168738 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:36:08.169608 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 556/1104] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:36:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:36:08.173583 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faf67b8-5531-476d-9380-459979c98314 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:08.221538 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:08.275624 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faf67b8-5531-476d-9380-459979c98314 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:08.287068 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a28b22-762d-4a86-9d93-ebe9ba90aaab None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:08.287639 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a28b22-762d-4a86-9d93-ebe9ba90aaab None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:08.287639 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a28b22-762d-4a86-9d93-ebe9ba90aaab None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:08.288920 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 557/1105] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:36:08.293989 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a45dc-f8c9-45e5-a962-39fbc58f2b70 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:08.294278 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a45dc-f8c9-45e5-a962-39fbc58f2b70 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:08.294556 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a45dc-f8c9-45e5-a962-39fbc58f2b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:08.310155 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae62575-5078-46e8-bf8d-139b3b962224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:08.310768 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae62575-5078-46e8-bf8d-139b3b962224 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:08.311048 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae62575-5078-46e8-bf8d-139b3b962224 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:08.311321 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae62575-5078-46e8-bf8d-139b3b962224 None admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:08.312009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae62575-5078-46e8-bf8d-139b3b962224 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:36:08.312316 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae62575-5078-46e8-bf8d-139b3b962224 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:08.312956 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-484a45dc-f8c9-45e5-a962-39fbc58f2b70 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:36:08.314035 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae62575-5078-46e8-bf8d-139b3b962224 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:08.314460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-484a45dc-f8c9-45e5-a962-39fbc58f2b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:36:08.326568 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 549/1106] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:36:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:08.332545 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae62575-5078-46e8-bf8d-139b3b962224 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:08.384304 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 558/1107] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:36:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:36:08.408222 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a45dc-f8c9-45e5-a962-39fbc58f2b70 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:08.432476 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae62575-5078-46e8-bf8d-139b3b962224 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:08.440228 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:08.467461 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a3568-4b2a-489d-a03c-30af4eaca0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:08.468072 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a3568-4b2a-489d-a03c-30af4eaca0b0 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:08.468347 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a3568-4b2a-489d-a03c-30af4eaca0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:08.468618 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a3568-4b2a-489d-a03c-30af4eaca0b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:08.469374 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356a3568-4b2a-489d-a03c-30af4eaca0b0 None admin] RBAC: Adding query filter params (role.id=ee973ce49dc142a6b0763794e078333d, scope.project.id=3caa2ea846f7449b8c483788fc5846c8, user.id=b22838a8ad3949e59cdbaaa1801cd07f) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:36:08.469685 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356a3568-4b2a-489d-a03c-30af4eaca0b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:08.471114 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356a3568-4b2a-489d-a03c-30af4eaca0b0 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:08.476827 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:08.480047 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 550/1108] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:36:08 2022] GET /identity/v3/role_assignments?user.id=b22838a8ad3949e59cdbaaa1801cd07f&scope.project.id=3caa2ea846f7449b8c483788fc5846c8&role.id=ee973ce49dc142a6b0763794e078333d => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:36:08.515111 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a45dc-f8c9-45e5-a962-39fbc58f2b70 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:08.551929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3dc8e-0f5e-4e9a-b66a-b9266321359f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:08.552590 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3dc8e-0f5e-4e9a-b66a-b9266321359f None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:08.552866 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3dc8e-0f5e-4e9a-b66a-b9266321359f None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:08.553144 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3dc8e-0f5e-4e9a-b66a-b9266321359f None admin] PATH_INFO: `/v3/domains/default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:08.554580 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d3dc8e-0f5e-4e9a-b66a-b9266321359f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:08.556064 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d3dc8e-0f5e-4e9a-b66a-b9266321359f None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:08.557740 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 559/1109] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:36:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:36:08.563972 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a3568-4b2a-489d-a03c-30af4eaca0b0 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:08.672006 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a3568-4b2a-489d-a03c-30af4eaca0b0 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:08.706348 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9fae81-4a6d-45ad-8970-11fcc71cab84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:08.706961 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fae81-4a6d-45ad-8970-11fcc71cab84 None admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:08.707237 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fae81-4a6d-45ad-8970-11fcc71cab84 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:08.707509 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fae81-4a6d-45ad-8970-11fcc71cab84 None admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:08.708146 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9fae81-4a6d-45ad-8970-11fcc71cab84 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:08.709673 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9fae81-4a6d-45ad-8970-11fcc71cab84 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:08.732516 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 551/1110] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 10 22:36:08 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:36:10.820058 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4c97c-eed8-4ff5-80fe-6dcce1703daf None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:10.820597 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4c97c-eed8-4ff5-80fe-6dcce1703daf None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:10.820702 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4c97c-eed8-4ff5-80fe-6dcce1703daf None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:10.821998 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 560/1111] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:36:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:36:10.827706 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5616dc5-7049-46a4-a8db-68d0253316ea None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:10.828118 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5616dc5-7049-46a4-a8db-68d0253316ea None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:10.828252 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5616dc5-7049-46a4-a8db-68d0253316ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:10.846331 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-f5616dc5-7049-46a4-a8db-68d0253316ea None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:36:10.847506 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-f5616dc5-7049-46a4-a8db-68d0253316ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:36:10.866860 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:10.917128 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 552/1112] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:36:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:36:10.940252 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce4c97c-eed8-4ff5-80fe-6dcce1703daf None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:11.040684 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce4c97c-eed8-4ff5-80fe-6dcce1703daf None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:11.073469 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d293026-29a3-436a-aed7-394629c2fabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:11.074026 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d293026-29a3-436a-aed7-394629c2fabd None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:11.074308 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d293026-29a3-436a-aed7-394629c2fabd None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:11.074582 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d293026-29a3-436a-aed7-394629c2fabd None admin] PATH_INFO: `/v3/roles/e22e4d2b3d06480bbcfa4d2ee3387c90` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:11.075958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d293026-29a3-436a-aed7-394629c2fabd None admin] RBAC: Authorizing `identity:get_role(role_id=e22e4d2b3d06480bbcfa4d2ee3387c90)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:11.077349 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d293026-29a3-436a-aed7-394629c2fabd None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:11.078644 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 561/1113] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:36:10 2022] GET /identity/v3/roles/e22e4d2b3d06480bbcfa4d2ee3387c90 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:11.084588 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5616dc5-7049-46a4-a8db-68d0253316ea None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:11.182352 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5616dc5-7049-46a4-a8db-68d0253316ea None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:11.223255 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39954ad5-6fac-4bec-b2d1-8952e4af8593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:11.223890 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39954ad5-6fac-4bec-b2d1-8952e4af8593 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:11.224167 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39954ad5-6fac-4bec-b2d1-8952e4af8593 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:11.224438 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39954ad5-6fac-4bec-b2d1-8952e4af8593 None admin] PATH_INFO: `/v3/users/dff1b7b7240649a0a6fd22e950fd9c39` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:11.237672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39954ad5-6fac-4bec-b2d1-8952e4af8593 None admin] RBAC: Authorizing `identity:get_user(user_id=dff1b7b7240649a0a6fd22e950fd9c39)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:11.239125 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39954ad5-6fac-4bec-b2d1-8952e4af8593 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:11.240770 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 553/1114] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:36:11 2022] GET /identity/v3/users/dff1b7b7240649a0a6fd22e950fd9c39 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:11.247282 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d293026-29a3-436a-aed7-394629c2fabd None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:11.250297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:11.345909 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d293026-29a3-436a-aed7-394629c2fabd None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:11.378143 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d1262-13cc-4116-9760-eb374bc360ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:11.378741 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d1262-13cc-4116-9760-eb374bc360ed None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:11.379022 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d1262-13cc-4116-9760-eb374bc360ed None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:11.379294 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d1262-13cc-4116-9760-eb374bc360ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:11.447090 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d1262-13cc-4116-9760-eb374bc360ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:11.448541 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d1262-13cc-4116-9760-eb374bc360ed None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:11.450372 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:11.558635 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 562/1115] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:36:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 22:36:11.565983 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39954ad5-6fac-4bec-b2d1-8952e4af8593 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:11.585225 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:11.670384 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39954ad5-6fac-4bec-b2d1-8952e4af8593 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:11.709425 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b9953e-9411-48a5-8a34-8da303225dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:11.710053 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9953e-9411-48a5-8a34-8da303225dba None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:11.710395 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9953e-9411-48a5-8a34-8da303225dba None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:11.710741 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9953e-9411-48a5-8a34-8da303225dba None admin] PATH_INFO: `/v3/projects/9e62fda33d2843ca9cd1614d910b1fb9` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:11.711824 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b9953e-9411-48a5-8a34-8da303225dba None admin] RBAC: Authorizing `identity:get_project(project_id=9e62fda33d2843ca9cd1614d910b1fb9)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:11.713246 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b9953e-9411-48a5-8a34-8da303225dba None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:11.714903 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 554/1116] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:36:11 2022] GET /identity/v3/projects/9e62fda33d2843ca9cd1614d910b1fb9 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:11.721028 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d1262-13cc-4116-9760-eb374bc360ed None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:11.820738 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d1262-13cc-4116-9760-eb374bc360ed None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:11.822317 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:11.855360 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a42fb-90af-4882-b73f-aca0cc228495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:11.855963 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a42fb-90af-4882-b73f-aca0cc228495 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:11.856276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a42fb-90af-4882-b73f-aca0cc228495 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:11.856549 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a42fb-90af-4882-b73f-aca0cc228495 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:11.857266 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7a42fb-90af-4882-b73f-aca0cc228495 None admin] RBAC: Adding query filter params (role.id=e22e4d2b3d06480bbcfa4d2ee3387c90, scope.project.id=9e62fda33d2843ca9cd1614d910b1fb9, user.id=dff1b7b7240649a0a6fd22e950fd9c39) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:36:11.857601 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7a42fb-90af-4882-b73f-aca0cc228495 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:11.859036 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7a42fb-90af-4882-b73f-aca0cc228495 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:11.864255 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 563/1117] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:36:11 2022] GET /identity/v3/role_assignments?user.id=dff1b7b7240649a0a6fd22e950fd9c39&scope.project.id=9e62fda33d2843ca9cd1614d910b1fb9&role.id=e22e4d2b3d06480bbcfa4d2ee3387c90 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:13.827466 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918906ac-00d6-4c5c-bb6b-5e667b1c3cf9 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:13.827993 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918906ac-00d6-4c5c-bb6b-5e667b1c3cf9 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:13.827993 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918906ac-00d6-4c5c-bb6b-5e667b1c3cf9 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:13.829236 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 555/1118] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:36:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:36:13.834394 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711b6ef-8f8f-4c55-877a-d69b727ae338 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:13.834829 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711b6ef-8f8f-4c55-877a-d69b727ae338 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:13.834935 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711b6ef-8f8f-4c55-877a-d69b727ae338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:13.853213 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-f711b6ef-8f8f-4c55-877a-d69b727ae338 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:36:13.854442 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-f711b6ef-8f8f-4c55-877a-d69b727ae338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:36:13.920439 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 564/1119] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:36:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:36:13.943246 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918906ac-00d6-4c5c-bb6b-5e667b1c3cf9 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:14.024429 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:14.043674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918906ac-00d6-4c5c-bb6b-5e667b1c3cf9 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:14.076751 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2c787-435e-4ab1-a315-6f9df50dd19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:14.077398 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2c787-435e-4ab1-a315-6f9df50dd19e None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:14.077723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2c787-435e-4ab1-a315-6f9df50dd19e None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:14.077938 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2c787-435e-4ab1-a315-6f9df50dd19e None admin] PATH_INFO: `/v3/roles/e22e4d2b3d06480bbcfa4d2ee3387c90` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:14.079294 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa2c787-435e-4ab1-a315-6f9df50dd19e None admin] RBAC: Authorizing `identity:get_role(role_id=e22e4d2b3d06480bbcfa4d2ee3387c90)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:14.080649 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa2c787-435e-4ab1-a315-6f9df50dd19e None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:14.081992 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 556/1120] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:36:13 2022] GET /identity/v3/roles/e22e4d2b3d06480bbcfa4d2ee3387c90 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:14.088690 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f711b6ef-8f8f-4c55-877a-d69b727ae338 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:14.179758 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:14.185896 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f711b6ef-8f8f-4c55-877a-d69b727ae338 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:14.219026 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c2896-46e2-46e1-b024-b8304a88f35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:14.219652 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c2896-46e2-46e1-b024-b8304a88f35d None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:14.219931 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c2896-46e2-46e1-b024-b8304a88f35d None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:14.220202 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c2896-46e2-46e1-b024-b8304a88f35d None admin] PATH_INFO: `/v3/users/dff1b7b7240649a0a6fd22e950fd9c39` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:14.221347 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8c2896-46e2-46e1-b024-b8304a88f35d None admin] RBAC: Authorizing `identity:get_user(user_id=dff1b7b7240649a0a6fd22e950fd9c39)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:14.222780 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8c2896-46e2-46e1-b024-b8304a88f35d None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:14.224346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 565/1121] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:36:14 2022] GET /identity/v3/users/dff1b7b7240649a0a6fd22e950fd9c39 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:14.230757 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2c787-435e-4ab1-a315-6f9df50dd19e None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:14.327569 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2c787-435e-4ab1-a315-6f9df50dd19e None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:14.361442 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd7f50-8048-4ade-a048-1819f66049f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:14.362071 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd7f50-8048-4ade-a048-1819f66049f7 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:14.362350 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd7f50-8048-4ade-a048-1819f66049f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:14.362622 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd7f50-8048-4ade-a048-1819f66049f7 None admin] PATH_INFO: `/v3/projects/9e62fda33d2843ca9cd1614d910b1fb9` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:14.363732 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd7f50-8048-4ade-a048-1819f66049f7 None admin] RBAC: Authorizing `identity:get_project(project_id=9e62fda33d2843ca9cd1614d910b1fb9)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:14.365159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd7f50-8048-4ade-a048-1819f66049f7 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:14.367026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 557/1122] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:36:14 2022] GET /identity/v3/projects/9e62fda33d2843ca9cd1614d910b1fb9 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:14.372996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c2896-46e2-46e1-b024-b8304a88f35d None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:14.474943 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c2896-46e2-46e1-b024-b8304a88f35d None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:14.507814 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c7b724-5479-4d8f-8aed-cae3401948f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:14.508458 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c7b724-5479-4d8f-8aed-cae3401948f5 None admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:14.508736 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c7b724-5479-4d8f-8aed-cae3401948f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:14.509010 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c7b724-5479-4d8f-8aed-cae3401948f5 None admin] PATH_INFO: `/v3/projects/9e62fda33d2843ca9cd1614d910b1fb9/users/dff1b7b7240649a0a6fd22e950fd9c39/roles/e22e4d2b3d06480bbcfa4d2ee3387c90` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:14.511178 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c7b724-5479-4d8f-8aed-cae3401948f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e62fda33d2843ca9cd1614d910b1fb9, user_id=dff1b7b7240649a0a6fd22e950fd9c39, role_id=e22e4d2b3d06480bbcfa4d2ee3387c90)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:14.512607 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c7b724-5479-4d8f-8aed-cae3401948f5 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:14.526015 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 566/1123] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 10 22:36:14 2022] PUT /identity/v3/projects/9e62fda33d2843ca9cd1614d910b1fb9/users/dff1b7b7240649a0a6fd22e950fd9c39/roles/e22e4d2b3d06480bbcfa4d2ee3387c90 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:36:16.570038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17518e63-ba04-4d46-ae44-7dbb6f632955 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:16.570591 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17518e63-ba04-4d46-ae44-7dbb6f632955 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:16.570591 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17518e63-ba04-4d46-ae44-7dbb6f632955 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:16.571942 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 558/1124] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:36:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:36:16.577207 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869f546-4dbb-4c21-b9f7-bb20f1c8b7a1 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:16.577603 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869f546-4dbb-4c21-b9f7-bb20f1c8b7a1 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:16.577781 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869f546-4dbb-4c21-b9f7-bb20f1c8b7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:16.595960 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-0869f546-4dbb-4c21-b9f7-bb20f1c8b7a1 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:36:16.597153 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-0869f546-4dbb-4c21-b9f7-bb20f1c8b7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:36:16.614861 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:16.668880 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 567/1125] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:36:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:36:16.692564 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17518e63-ba04-4d46-ae44-7dbb6f632955 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:16.712418 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:16.795244 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17518e63-ba04-4d46-ae44-7dbb6f632955 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:16.828205 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4644cb11-3a82-4be3-a639-2c36cffd8989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:16.828813 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4644cb11-3a82-4be3-a639-2c36cffd8989 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:16.829085 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4644cb11-3a82-4be3-a639-2c36cffd8989 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:16.829380 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4644cb11-3a82-4be3-a639-2c36cffd8989 None admin] PATH_INFO: `/v3/roles/e22e4d2b3d06480bbcfa4d2ee3387c90` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:16.830738 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4644cb11-3a82-4be3-a639-2c36cffd8989 None admin] RBAC: Authorizing `identity:get_role(role_id=e22e4d2b3d06480bbcfa4d2ee3387c90)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:16.832099 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4644cb11-3a82-4be3-a639-2c36cffd8989 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:16.833416 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 559/1126] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:36:16 2022] GET /identity/v3/roles/e22e4d2b3d06480bbcfa4d2ee3387c90 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:16.840189 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0869f546-4dbb-4c21-b9f7-bb20f1c8b7a1 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:16.937780 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0869f546-4dbb-4c21-b9f7-bb20f1c8b7a1 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:16.971561 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0b076-10da-4b67-8f13-5a1987be3b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:16.972122 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0b076-10da-4b67-8f13-5a1987be3b06 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:16.972403 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0b076-10da-4b67-8f13-5a1987be3b06 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:16.972678 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0b076-10da-4b67-8f13-5a1987be3b06 None admin] PATH_INFO: `/v3/users/dff1b7b7240649a0a6fd22e950fd9c39` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:16.973816 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0b076-10da-4b67-8f13-5a1987be3b06 None admin] RBAC: Authorizing `identity:get_user(user_id=dff1b7b7240649a0a6fd22e950fd9c39)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:16.975253 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0b076-10da-4b67-8f13-5a1987be3b06 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:16.977351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 568/1127] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:36:16 2022] GET /identity/v3/users/dff1b7b7240649a0a6fd22e950fd9c39 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:16.984250 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4644cb11-3a82-4be3-a639-2c36cffd8989 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:17.070169 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:17.085502 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4644cb11-3a82-4be3-a639-2c36cffd8989 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:17.118762 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b645e903-27db-4011-b38d-3b013468839f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:17.119378 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645e903-27db-4011-b38d-3b013468839f None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:17.119641 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645e903-27db-4011-b38d-3b013468839f None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:17.119912 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645e903-27db-4011-b38d-3b013468839f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:17.184426 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b645e903-27db-4011-b38d-3b013468839f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:17.185905 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b645e903-27db-4011-b38d-3b013468839f None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:17.265702 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:17.281211 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 560/1128] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:36:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 22:36:17.287160 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0b076-10da-4b67-8f13-5a1987be3b06 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:17.311877 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:17.385899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0b076-10da-4b67-8f13-5a1987be3b06 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:17.420005 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cf431b-47a0-4f15-9156-6ac0abdf4fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:17.420637 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf431b-47a0-4f15-9156-6ac0abdf4fb5 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:17.420919 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf431b-47a0-4f15-9156-6ac0abdf4fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:17.421192 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf431b-47a0-4f15-9156-6ac0abdf4fb5 None admin] PATH_INFO: `/v3/projects/9e62fda33d2843ca9cd1614d910b1fb9` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:17.422337 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cf431b-47a0-4f15-9156-6ac0abdf4fb5 None admin] RBAC: Authorizing `identity:get_project(project_id=9e62fda33d2843ca9cd1614d910b1fb9)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:17.423769 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cf431b-47a0-4f15-9156-6ac0abdf4fb5 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:17.425604 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 569/1129] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:36:17 2022] GET /identity/v3/projects/9e62fda33d2843ca9cd1614d910b1fb9 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:17.431503 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b645e903-27db-4011-b38d-3b013468839f None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:17.529039 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b645e903-27db-4011-b38d-3b013468839f None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:17.561821 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76dddea-2f14-4dc2-93fc-849ad48e2875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:17.562412 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76dddea-2f14-4dc2-93fc-849ad48e2875 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:17.562694 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76dddea-2f14-4dc2-93fc-849ad48e2875 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:17.562962 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76dddea-2f14-4dc2-93fc-849ad48e2875 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:17.563935 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76dddea-2f14-4dc2-93fc-849ad48e2875 None admin] RBAC: Adding query filter params (role.id=e22e4d2b3d06480bbcfa4d2ee3387c90, scope.project.id=9e62fda33d2843ca9cd1614d910b1fb9, user.id=dff1b7b7240649a0a6fd22e950fd9c39) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:36:17.564236 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76dddea-2f14-4dc2-93fc-849ad48e2875 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:17.565686 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76dddea-2f14-4dc2-93fc-849ad48e2875 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:17.572061 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 561/1130] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:36:17 2022] GET /identity/v3/role_assignments?user.id=dff1b7b7240649a0a6fd22e950fd9c39&scope.project.id=9e62fda33d2843ca9cd1614d910b1fb9&role.id=e22e4d2b3d06480bbcfa4d2ee3387c90 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:19.593035 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237b66f-9d2b-4a08-9e7c-cd24e1098883 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:19.593618 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237b66f-9d2b-4a08-9e7c-cd24e1098883 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:19.593618 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237b66f-9d2b-4a08-9e7c-cd24e1098883 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:19.594952 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 570/1131] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:36:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:36:19.599987 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c19f5a-ba7b-4081-9173-8f65199e6a2e None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:19.600451 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c19f5a-ba7b-4081-9173-8f65199e6a2e None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:19.600544 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c19f5a-ba7b-4081-9173-8f65199e6a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:19.619014 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-37c19f5a-ba7b-4081-9173-8f65199e6a2e None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:36:19.620198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-37c19f5a-ba7b-4081-9173-8f65199e6a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:36:19.664940 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:19.687036 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 562/1132] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:36:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:36:19.710768 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237b66f-9d2b-4a08-9e7c-cd24e1098883 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:19.807983 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:19.819169 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237b66f-9d2b-4a08-9e7c-cd24e1098883 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:19.853129 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d041fc-9860-4d0d-b4c4-ec99eff2a5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:19.853785 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d041fc-9860-4d0d-b4c4-ec99eff2a5e0 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:19.854066 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d041fc-9860-4d0d-b4c4-ec99eff2a5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:19.854342 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d041fc-9860-4d0d-b4c4-ec99eff2a5e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:19.855457 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d041fc-9860-4d0d-b4c4-ec99eff2a5e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:19.856894 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d041fc-9860-4d0d-b4c4-ec99eff2a5e0 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:19.858494 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 571/1133] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:36:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:19.864380 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c19f5a-ba7b-4081-9173-8f65199e6a2e None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:19.961379 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c19f5a-ba7b-4081-9173-8f65199e6a2e None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:19.995478 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5f0be-fde3-4e58-98d4-b10eff149be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:19.996074 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5f0be-fde3-4e58-98d4-b10eff149be4 None admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:19.996348 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5f0be-fde3-4e58-98d4-b10eff149be4 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:19.996613 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5f0be-fde3-4e58-98d4-b10eff149be4 None admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:19.997233 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5f0be-fde3-4e58-98d4-b10eff149be4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:19.998685 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5f0be-fde3-4e58-98d4-b10eff149be4 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:20.024498 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 563/1134] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 10 22:36:19 2022] POST /identity/v3/projects => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:36:22.063046 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424757d7-c519-4caf-9677-77d009a2f978 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:22.063583 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424757d7-c519-4caf-9677-77d009a2f978 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:22.063583 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424757d7-c519-4caf-9677-77d009a2f978 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:22.064893 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 572/1135] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:36:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:36:22.070197 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55298118-f3bd-4abb-9665-42e320a1d6cb None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:22.070620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55298118-f3bd-4abb-9665-42e320a1d6cb None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:22.070755 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55298118-f3bd-4abb-9665-42e320a1d6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:22.088650 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-55298118-f3bd-4abb-9665-42e320a1d6cb None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:36:22.089881 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-55298118-f3bd-4abb-9665-42e320a1d6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:36:22.115538 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:22.157382 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 564/1136] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:36:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:36:22.181393 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424757d7-c519-4caf-9677-77d009a2f978 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:22.281754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424757d7-c519-4caf-9677-77d009a2f978 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:22.314615 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef71bc98-bec2-44cc-908b-3984dcd4b89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:22.315163 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71bc98-bec2-44cc-908b-3984dcd4b89a None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:22.315443 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71bc98-bec2-44cc-908b-3984dcd4b89a None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:22.315712 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71bc98-bec2-44cc-908b-3984dcd4b89a None admin] PATH_INFO: `/v3/domains/default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:22.316801 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef71bc98-bec2-44cc-908b-3984dcd4b89a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:22.318273 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef71bc98-bec2-44cc-908b-3984dcd4b89a None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:22.320508 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 573/1137] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:36:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:22.326353 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55298118-f3bd-4abb-9665-42e320a1d6cb None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:22.424455 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55298118-f3bd-4abb-9665-42e320a1d6cb None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:22.459635 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ed356-7995-4ff7-9c06-007e25ed6e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:22.460248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ed356-7995-4ff7-9c06-007e25ed6e91 None admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:22.460530 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ed356-7995-4ff7-9c06-007e25ed6e91 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:22.460803 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ed356-7995-4ff7-9c06-007e25ed6e91 None admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:22.461484 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ed356-7995-4ff7-9c06-007e25ed6e91 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:22.462916 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ed356-7995-4ff7-9c06-007e25ed6e91 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:22.483435 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 565/1138] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 10 22:36:22 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:36:24.539493 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb5846-8b93-482e-aea2-b76e3d2ef46c None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:24.540028 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb5846-8b93-482e-aea2-b76e3d2ef46c None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:24.540199 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb5846-8b93-482e-aea2-b76e3d2ef46c None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:24.541987 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 574/1139] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:36:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:36:24.547237 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4b23f-3c6e-4c73-87fa-12f75c001a67 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:24.547682 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4b23f-3c6e-4c73-87fa-12f75c001a67 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:24.547783 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4b23f-3c6e-4c73-87fa-12f75c001a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:24.566344 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-2da4b23f-3c6e-4c73-87fa-12f75c001a67 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:36:24.567523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-2da4b23f-3c6e-4c73-87fa-12f75c001a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:36:24.580282 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:24.643928 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 566/1140] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:36:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:36:24.668613 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bb5846-8b93-482e-aea2-b76e3d2ef46c None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:24.697363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:24.773242 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bb5846-8b93-482e-aea2-b76e3d2ef46c None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:24.806541 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628c9d4d-91c0-442b-9970-4fc975835fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:24.807170 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c9d4d-91c0-442b-9970-4fc975835fd2 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:24.807452 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c9d4d-91c0-442b-9970-4fc975835fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:24.807727 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c9d4d-91c0-442b-9970-4fc975835fd2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:24.817959 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628c9d4d-91c0-442b-9970-4fc975835fd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:36:24.822641 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-628c9d4d-91c0-442b-9970-4fc975835fd2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:36:24.822641 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:36:24.822641 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:36:24.822641 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:36:24.822641 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:36:24.822641 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:36:24.822641 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:36:24.822641 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:36:24.822641 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:36:24.822641 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:36:24.822641 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:36:24.822641 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:36:24.822641 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:36:24.822641 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:36:24.822641 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:36:24.822641 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:36:24.823830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:36:24.823830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:36:24.823830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:36:24.823830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:36:24.823830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:36:24.823830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:36:24.823830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:36:24.823830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:36:24.823830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:36:24.823830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:36:24.823830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:36:24.823830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:36:24.823830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:36:24.823830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:36:24.823830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:36:24.823830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:36:24.825118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:36:24.825118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:36:24.825118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:36:24.825118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:36:24.825118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:36:24.825118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:36:24.825118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:36:24.825118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:36:24.825118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:36:24.825118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:36:24.825118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:36:24.825118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:36:24.825118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 575/1141] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:36:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:36:24.829847 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4b23f-3c6e-4c73-87fa-12f75c001a67 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:24.929765 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4b23f-3c6e-4c73-87fa-12f75c001a67 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:24.964382 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100675f-e153-44d8-93f1-4eebfcdf6cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:24.965188 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100675f-e153-44d8-93f1-4eebfcdf6cdb None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:24.965638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100675f-e153-44d8-93f1-4eebfcdf6cdb None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:24.966042 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100675f-e153-44d8-93f1-4eebfcdf6cdb None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:24.967030 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9100675f-e153-44d8-93f1-4eebfcdf6cdb None admin] RBAC: Adding query filter params (name=admin) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:36:24.967486 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9100675f-e153-44d8-93f1-4eebfcdf6cdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:24.969642 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9100675f-e153-44d8-93f1-4eebfcdf6cdb None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:24.981273 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 567/1142] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:36:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:24.987097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628c9d4d-91c0-442b-9970-4fc975835fd2 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:25.079875 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:25.095453 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628c9d4d-91c0-442b-9970-4fc975835fd2 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:25.128501 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073f5bf-9bc2-40e1-aea1-254e66296c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:25.129127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073f5bf-9bc2-40e1-aea1-254e66296c93 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:25.129462 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073f5bf-9bc2-40e1-aea1-254e66296c93 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:25.129732 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073f5bf-9bc2-40e1-aea1-254e66296c93 None admin] PATH_INFO: `/v3/users/7154e30a412e497289ae8cf40ce5d564` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:25.143266 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073f5bf-9bc2-40e1-aea1-254e66296c93 None admin] RBAC: Authorizing `identity:get_user(user_id=7154e30a412e497289ae8cf40ce5d564)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:25.144707 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073f5bf-9bc2-40e1-aea1-254e66296c93 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:25.146314 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 576/1143] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:36:24 2022] GET /identity/v3/users/7154e30a412e497289ae8cf40ce5d564 => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:25.152748 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100675f-e153-44d8-93f1-4eebfcdf6cdb None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:25.165179 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:25.250857 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100675f-e153-44d8-93f1-4eebfcdf6cdb None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:25.284960 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c9595-7043-48bc-8271-bd7c690b6604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:25.285583 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c9595-7043-48bc-8271-bd7c690b6604 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:25.285864 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c9595-7043-48bc-8271-bd7c690b6604 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:25.286136 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c9595-7043-48bc-8271-bd7c690b6604 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:25.354874 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7c9595-7043-48bc-8271-bd7c690b6604 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:25.356323 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7c9595-7043-48bc-8271-bd7c690b6604 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:25.367711 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:25.454032 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 568/1144] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:36:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 22:36:25.460069 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073f5bf-9bc2-40e1-aea1-254e66296c93 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:25.556691 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073f5bf-9bc2-40e1-aea1-254e66296c93 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:25.589764 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de06e1-9070-4239-82c7-2faa47ca68a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:25.590388 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de06e1-9070-4239-82c7-2faa47ca68a2 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:25.590669 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de06e1-9070-4239-82c7-2faa47ca68a2 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:25.590941 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de06e1-9070-4239-82c7-2faa47ca68a2 None admin] PATH_INFO: `/v3/projects/aa4c16f6b7ce411c89e02815cee91f1a` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:25.592046 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44de06e1-9070-4239-82c7-2faa47ca68a2 None admin] RBAC: Authorizing `identity:get_project(project_id=aa4c16f6b7ce411c89e02815cee91f1a)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:25.593498 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44de06e1-9070-4239-82c7-2faa47ca68a2 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:25.595100 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 577/1145] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:36:25 2022] GET /identity/v3/projects/aa4c16f6b7ce411c89e02815cee91f1a => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:25.601302 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c9595-7043-48bc-8271-bd7c690b6604 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:25.699498 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c9595-7043-48bc-8271-bd7c690b6604 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:25.709968 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:25.733434 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3435f7-123d-43af-833b-773bfe2797a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:25.734036 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3435f7-123d-43af-833b-773bfe2797a1 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:25.734316 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3435f7-123d-43af-833b-773bfe2797a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:25.734591 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3435f7-123d-43af-833b-773bfe2797a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:25.735306 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3435f7-123d-43af-833b-773bfe2797a1 None admin] RBAC: Adding query filter params (role.id=6641e95fdc1f4c098ebc372bb692d227, scope.project.id=aa4c16f6b7ce411c89e02815cee91f1a, user.id=7154e30a412e497289ae8cf40ce5d564) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:36:25.735613 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3435f7-123d-43af-833b-773bfe2797a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:25.737041 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3435f7-123d-43af-833b-773bfe2797a1 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:25.743174 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 569/1146] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:36:25 2022] GET /identity/v3/role_assignments?user.id=7154e30a412e497289ae8cf40ce5d564&scope.project.id=aa4c16f6b7ce411c89e02815cee91f1a&role.id=6641e95fdc1f4c098ebc372bb692d227 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:27.749793 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b7aa6-45fb-4663-b085-ebdc1501935b None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:27.750305 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b7aa6-45fb-4663-b085-ebdc1501935b None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:27.750305 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b7aa6-45fb-4663-b085-ebdc1501935b None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:27.751573 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 578/1147] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:36:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:36:27.757591 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79317676-0f32-45c4-8627-5d04a4c19b87 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:27.758094 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79317676-0f32-45c4-8627-5d04a4c19b87 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:27.758191 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79317676-0f32-45c4-8627-5d04a4c19b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:27.775872 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-79317676-0f32-45c4-8627-5d04a4c19b87 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:36:27.777073 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-79317676-0f32-45c4-8627-5d04a4c19b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:36:27.849523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 570/1148] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:36:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:36:27.872248 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b7aa6-45fb-4663-b085-ebdc1501935b None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:27.887839 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:27.975667 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b7aa6-45fb-4663-b085-ebdc1501935b None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:28.009070 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49e9f6-2987-4431-8828-a225f3dfc0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:28.009663 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49e9f6-2987-4431-8828-a225f3dfc0fe None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:28.009949 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49e9f6-2987-4431-8828-a225f3dfc0fe None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:28.010225 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49e9f6-2987-4431-8828-a225f3dfc0fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:28.020078 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49e9f6-2987-4431-8828-a225f3dfc0fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:36:28.025004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-4d49e9f6-2987-4431-8828-a225f3dfc0fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:36:28.025004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:36:28.025004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:36:28.025004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:36:28.025004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:36:28.025004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:36:28.025004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:36:28.025004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:36:28.025004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:36:28.025004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:36:28.025004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:36:28.025004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:36:28.025004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:36:28.025004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:36:28.025004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:36:28.025004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:36:28.026173 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:36:28.026173 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:36:28.026173 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:36:28.026173 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:36:28.026173 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:36:28.026173 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:36:28.026173 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:36:28.026173 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:36:28.026173 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:36:28.026173 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:36:28.026173 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:36:28.026173 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:36:28.026173 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:36:28.026173 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:36:28.026173 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:36:28.026173 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:36:28.027008 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:36:28.027008 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:36:28.027008 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:36:28.027008 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:36:28.027008 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:36:28.027008 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:36:28.027008 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:36:28.027008 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:36:28.027008 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:36:28.027008 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:36:28.027008 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:36:28.027008 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:36:28.027008 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 579/1149] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:36:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:36:28.032277 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79317676-0f32-45c4-8627-5d04a4c19b87 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:28.132695 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79317676-0f32-45c4-8627-5d04a4c19b87 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:28.135018 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:28.168115 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91462d14-aaa2-4fc5-85f1-b89ebd066916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:28.168714 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91462d14-aaa2-4fc5-85f1-b89ebd066916 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:28.168995 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91462d14-aaa2-4fc5-85f1-b89ebd066916 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:28.169265 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91462d14-aaa2-4fc5-85f1-b89ebd066916 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:28.169951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91462d14-aaa2-4fc5-85f1-b89ebd066916 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:36:28.170250 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91462d14-aaa2-4fc5-85f1-b89ebd066916 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:28.171604 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91462d14-aaa2-4fc5-85f1-b89ebd066916 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:28.180438 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 571/1150] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:36:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:28.186442 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49e9f6-2987-4431-8828-a225f3dfc0fe None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:28.255459 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:28.284478 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49e9f6-2987-4431-8828-a225f3dfc0fe None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:28.317268 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300508b6-aaca-411e-8bfb-f57ef3a49516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:28.317903 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300508b6-aaca-411e-8bfb-f57ef3a49516 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:28.318268 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300508b6-aaca-411e-8bfb-f57ef3a49516 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:28.318561 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300508b6-aaca-411e-8bfb-f57ef3a49516 None admin] PATH_INFO: `/v3/users/7154e30a412e497289ae8cf40ce5d564` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:28.319675 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300508b6-aaca-411e-8bfb-f57ef3a49516 None admin] RBAC: Authorizing `identity:get_user(user_id=7154e30a412e497289ae8cf40ce5d564)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:28.321092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300508b6-aaca-411e-8bfb-f57ef3a49516 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:28.322667 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 580/1151] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:36:28 2022] GET /identity/v3/users/7154e30a412e497289ae8cf40ce5d564 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:28.328438 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91462d14-aaa2-4fc5-85f1-b89ebd066916 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:28.429133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91462d14-aaa2-4fc5-85f1-b89ebd066916 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:28.463064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de377777-9588-4c1e-9cbb-b9c27af31e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:28.463691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de377777-9588-4c1e-9cbb-b9c27af31e3a None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:28.463971 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de377777-9588-4c1e-9cbb-b9c27af31e3a None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:28.464243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de377777-9588-4c1e-9cbb-b9c27af31e3a None admin] PATH_INFO: `/v3/projects/aa4c16f6b7ce411c89e02815cee91f1a` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:28.465386 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de377777-9588-4c1e-9cbb-b9c27af31e3a None admin] RBAC: Authorizing `identity:get_project(project_id=aa4c16f6b7ce411c89e02815cee91f1a)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:28.466818 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de377777-9588-4c1e-9cbb-b9c27af31e3a None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:28.468457 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 572/1152] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:36:28 2022] GET /identity/v3/projects/aa4c16f6b7ce411c89e02815cee91f1a => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:28.474367 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300508b6-aaca-411e-8bfb-f57ef3a49516 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:28.572703 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300508b6-aaca-411e-8bfb-f57ef3a49516 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:28.605882 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd3f95-68d1-4ea9-8ef0-a4984dff2e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:28.606533 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd3f95-68d1-4ea9-8ef0-a4984dff2e9a None admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:28.606816 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd3f95-68d1-4ea9-8ef0-a4984dff2e9a None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:28.607092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd3f95-68d1-4ea9-8ef0-a4984dff2e9a None admin] PATH_INFO: `/v3/projects/aa4c16f6b7ce411c89e02815cee91f1a/users/7154e30a412e497289ae8cf40ce5d564/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:28.608966 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fd3f95-68d1-4ea9-8ef0-a4984dff2e9a None admin] RBAC: Authorizing `identity:create_grant(project_id=aa4c16f6b7ce411c89e02815cee91f1a, user_id=7154e30a412e497289ae8cf40ce5d564, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:28.610435 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fd3f95-68d1-4ea9-8ef0-a4984dff2e9a None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:28.619175 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:28.624139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 581/1153] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 10 22:36:28 2022] PUT /identity/v3/projects/aa4c16f6b7ce411c89e02815cee91f1a/users/7154e30a412e497289ae8cf40ce5d564/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:36:30.586587 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6417708-3745-4bee-9270-3b911dc4fae5 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:30.587110 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6417708-3745-4bee-9270-3b911dc4fae5 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:30.587110 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6417708-3745-4bee-9270-3b911dc4fae5 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:30.588474 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 573/1154] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:36:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:36:30.593701 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121be31-ede4-4a41-acde-198e786a90c6 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:30.594126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121be31-ede4-4a41-acde-198e786a90c6 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:30.594247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121be31-ede4-4a41-acde-198e786a90c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:30.611790 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-c121be31-ede4-4a41-acde-198e786a90c6 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:36:30.612978 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-c121be31-ede4-4a41-acde-198e786a90c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:36:30.687879 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 582/1155] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:36:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:36:30.712030 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6417708-3745-4bee-9270-3b911dc4fae5 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:30.747115 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:30.812304 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6417708-3745-4bee-9270-3b911dc4fae5 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:30.844248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf302377-dbb2-40c6-8263-6ba4286a8456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:30.844858 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf302377-dbb2-40c6-8263-6ba4286a8456 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:30.845132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf302377-dbb2-40c6-8263-6ba4286a8456 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:30.845575 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf302377-dbb2-40c6-8263-6ba4286a8456 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:30.855736 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf302377-dbb2-40c6-8263-6ba4286a8456 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:36:30.860262 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-bf302377-dbb2-40c6-8263-6ba4286a8456 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:36:30.860262 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:36:30.860262 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:36:30.860262 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:36:30.860262 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:36:30.860262 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:36:30.860262 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:36:30.860262 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:36:30.860262 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:36:30.860262 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:36:30.860262 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:36:30.860262 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:36:30.860262 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:36:30.860262 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:36:30.860262 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:36:30.860262 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:36:30.861431 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:36:30.861431 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:36:30.861431 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:36:30.861431 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:36:30.861431 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:36:30.861431 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:36:30.861431 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:36:30.861431 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:36:30.861431 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:36:30.861431 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:36:30.861431 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:36:30.861431 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:36:30.861431 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:36:30.861431 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:36:30.861431 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:36:30.861431 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:36:30.862440 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:36:30.862440 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:36:30.862440 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:36:30.862440 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:36:30.862440 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:36:30.862440 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:36:30.862440 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:36:30.862440 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:36:30.862440 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:36:30.862440 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:36:30.862440 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:36:30.862440 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:36:30.862440 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 574/1156] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:36:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:36:30.868277 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121be31-ede4-4a41-acde-198e786a90c6 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:30.966346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121be31-ede4-4a41-acde-198e786a90c6 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:30.995685 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:31.001427 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d79e9-94bb-47c1-ab13-494e92dda758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:31.002038 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d79e9-94bb-47c1-ab13-494e92dda758 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:31.002317 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d79e9-94bb-47c1-ab13-494e92dda758 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:31.002587 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d79e9-94bb-47c1-ab13-494e92dda758 None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:31.003240 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5d79e9-94bb-47c1-ab13-494e92dda758 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:36:31.003540 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5d79e9-94bb-47c1-ab13-494e92dda758 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:31.004922 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5d79e9-94bb-47c1-ab13-494e92dda758 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:31.013451 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 583/1157] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:36:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:31.019274 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf302377-dbb2-40c6-8263-6ba4286a8456 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:31.112769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:31.120367 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf302377-dbb2-40c6-8263-6ba4286a8456 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:31.153338 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149b078b-5c9a-4a9b-b35a-3d40dcde21f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:31.153956 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b078b-5c9a-4a9b-b35a-3d40dcde21f6 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:31.154240 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b078b-5c9a-4a9b-b35a-3d40dcde21f6 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:31.154515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b078b-5c9a-4a9b-b35a-3d40dcde21f6 None admin] PATH_INFO: `/v3/users/7154e30a412e497289ae8cf40ce5d564` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:31.155627 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149b078b-5c9a-4a9b-b35a-3d40dcde21f6 None admin] RBAC: Authorizing `identity:get_user(user_id=7154e30a412e497289ae8cf40ce5d564)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:31.157054 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149b078b-5c9a-4a9b-b35a-3d40dcde21f6 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:31.158666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 575/1158] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:36:31 2022] GET /identity/v3/users/7154e30a412e497289ae8cf40ce5d564 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:31.164978 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d79e9-94bb-47c1-ab13-494e92dda758 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:31.262199 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d79e9-94bb-47c1-ab13-494e92dda758 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:31.295111 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e215c3c-e263-4b61-8ed6-8d970d26ae60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:31.295706 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e215c3c-e263-4b61-8ed6-8d970d26ae60 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:31.295983 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e215c3c-e263-4b61-8ed6-8d970d26ae60 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:31.296250 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e215c3c-e263-4b61-8ed6-8d970d26ae60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:31.354238 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:31.363659 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e215c3c-e263-4b61-8ed6-8d970d26ae60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:31.365158 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e215c3c-e263-4b61-8ed6-8d970d26ae60 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:31.461780 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 584/1159] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:36:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 22:36:31.467678 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149b078b-5c9a-4a9b-b35a-3d40dcde21f6 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:31.571273 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149b078b-5c9a-4a9b-b35a-3d40dcde21f6 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:31.603518 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a3cbb-7169-47b3-b517-ec3a71a1b810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:31.604141 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a3cbb-7169-47b3-b517-ec3a71a1b810 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:31.604421 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a3cbb-7169-47b3-b517-ec3a71a1b810 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:31.604695 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a3cbb-7169-47b3-b517-ec3a71a1b810 None admin] PATH_INFO: `/v3/projects/aa4c16f6b7ce411c89e02815cee91f1a` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:31.605831 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23a3cbb-7169-47b3-b517-ec3a71a1b810 None admin] RBAC: Authorizing `identity:get_project(project_id=aa4c16f6b7ce411c89e02815cee91f1a)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:31.607269 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23a3cbb-7169-47b3-b517-ec3a71a1b810 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:31.608887 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 576/1160] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:36:31 2022] GET /identity/v3/projects/aa4c16f6b7ce411c89e02815cee91f1a => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:31.615672 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e215c3c-e263-4b61-8ed6-8d970d26ae60 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:31.704249 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:31.713235 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e215c3c-e263-4b61-8ed6-8d970d26ae60 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:31.746822 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f07493-5077-4fa7-a4ba-d9b6cc3f4cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:31.747416 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f07493-5077-4fa7-a4ba-d9b6cc3f4cf2 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:31.747692 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f07493-5077-4fa7-a4ba-d9b6cc3f4cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:31.747961 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f07493-5077-4fa7-a4ba-d9b6cc3f4cf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:31.748666 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f07493-5077-4fa7-a4ba-d9b6cc3f4cf2 None admin] RBAC: Adding query filter params (role.id=6641e95fdc1f4c098ebc372bb692d227, scope.project.id=aa4c16f6b7ce411c89e02815cee91f1a, user.id=7154e30a412e497289ae8cf40ce5d564) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:36:31.748966 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f07493-5077-4fa7-a4ba-d9b6cc3f4cf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:31.750423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f07493-5077-4fa7-a4ba-d9b6cc3f4cf2 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:31.756144 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 585/1161] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:36:31 2022] GET /identity/v3/role_assignments?user.id=7154e30a412e497289ae8cf40ce5d564&scope.project.id=aa4c16f6b7ce411c89e02815cee91f1a&role.id=6641e95fdc1f4c098ebc372bb692d227 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:33.829951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca158d-6834-4af0-98ca-8898274f0b3f None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:33.830483 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca158d-6834-4af0-98ca-8898274f0b3f None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:33.830483 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca158d-6834-4af0-98ca-8898274f0b3f None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:33.831741 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 577/1162] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:36:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:36:33.840682 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17703e55-7f93-4d57-9fe5-7190affe65ca None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:33.841131 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17703e55-7f93-4d57-9fe5-7190affe65ca None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:33.841243 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17703e55-7f93-4d57-9fe5-7190affe65ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:33.859802 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-17703e55-7f93-4d57-9fe5-7190affe65ca None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:36:33.860997 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-17703e55-7f93-4d57-9fe5-7190affe65ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:36:33.925772 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 586/1163] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:36:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:36:33.948078 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca158d-6834-4af0-98ca-8898274f0b3f None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:33.978319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:34.048613 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca158d-6834-4af0-98ca-8898274f0b3f None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:34.080842 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743185c4-20b1-4d28-87bb-c2f2609503ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:34.081391 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743185c4-20b1-4d28-87bb-c2f2609503ff None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:34.081667 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743185c4-20b1-4d28-87bb-c2f2609503ff None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:34.081937 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743185c4-20b1-4d28-87bb-c2f2609503ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:34.147641 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743185c4-20b1-4d28-87bb-c2f2609503ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:34.149068 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743185c4-20b1-4d28-87bb-c2f2609503ff None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:34.177126 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:34.245437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 578/1164] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:36:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 22:36:34.251320 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17703e55-7f93-4d57-9fe5-7190affe65ca None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:34.332259 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:34.349827 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17703e55-7f93-4d57-9fe5-7190affe65ca None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:34.382296 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d5b616-8fb2-4de1-a0be-f8aa5ff1a5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:34.382919 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5b616-8fb2-4de1-a0be-f8aa5ff1a5d4 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:34.383192 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5b616-8fb2-4de1-a0be-f8aa5ff1a5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:34.383459 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5b616-8fb2-4de1-a0be-f8aa5ff1a5d4 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:34.389822 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d5b616-8fb2-4de1-a0be-f8aa5ff1a5d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:34.391245 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d5b616-8fb2-4de1-a0be-f8aa5ff1a5d4 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:34.402796 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-b4d5b616-8fb2-4de1-a0be-f8aa5ff1a5d4 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 10 22:36:34.402796 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:36:34.402796 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 10 22:36:34.402796 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 10 22:36:34.402796 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:36:34.402796 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:36:34.402796 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:36:34.402796 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:36:34.402796 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 10 22:36:34.402796 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:36:34.402796 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:36:34.402796 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:36:34.402796 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:36:34.402796 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:36:34.402796 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:36:34.402796 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:36:34.402796 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:36:34.404031 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:36:34.404031 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:36:34.404031 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:36:34.404031 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:36:34.404031 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:36:34.404031 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:36:34.404031 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:36:34.404031 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:36:34.404031 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:36:34.404031 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:36:34.404031 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:36:34.404031 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:36:34.404031 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:36:34.404031 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:36:34.404031 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:36:34.404031 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:36:34.405241 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:36:34.405241 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:36:34.405241 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:36:34.405241 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:36:34.405241 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:36:34.405241 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:36:34.405241 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:36:34.405241 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:36:34.405241 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:36:34.405241 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:36:34.405241 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 10 22:36:34.405241 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:36:34.405241 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 10 22:36:34.405241 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:36:34.405241 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 587/1165] 10.222.0.76 () {56 vars in 1117 bytes} [Wed Aug 10 22:36:34 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:36:34.409932 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743185c4-20b1-4d28-87bb-c2f2609503ff None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:34.507685 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743185c4-20b1-4d28-87bb-c2f2609503ff None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:34.535238 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:34.542012 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becdd53a-ee05-4b65-8d7a-1c54172eed5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:34.542616 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdd53a-ee05-4b65-8d7a-1c54172eed5e None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:34.542896 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdd53a-ee05-4b65-8d7a-1c54172eed5e None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:34.543174 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdd53a-ee05-4b65-8d7a-1c54172eed5e None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:34.543808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becdd53a-ee05-4b65-8d7a-1c54172eed5e None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:36:34.544107 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becdd53a-ee05-4b65-8d7a-1c54172eed5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:34.545495 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becdd53a-ee05-4b65-8d7a-1c54172eed5e None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:34.552592 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 579/1166] 10.222.0.76 () {56 vars in 1115 bytes} [Wed Aug 10 22:36:34 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:36.584225 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c6216-1ea4-4b29-8803-29e7e94e5008 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:36.584753 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c6216-1ea4-4b29-8803-29e7e94e5008 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:36.584753 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c6216-1ea4-4b29-8803-29e7e94e5008 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:36.586138 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 588/1167] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:36:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:36:36.591269 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b444750-4ab8-4e29-93f6-5e87d0bee5de None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:36.591652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b444750-4ab8-4e29-93f6-5e87d0bee5de None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:36.591913 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b444750-4ab8-4e29-93f6-5e87d0bee5de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:36.610562 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-3b444750-4ab8-4e29-93f6-5e87d0bee5de None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:36:36.611721 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-3b444750-4ab8-4e29-93f6-5e87d0bee5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:36:36.678190 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 580/1168] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:36:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:36:36.700532 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c6216-1ea4-4b29-8803-29e7e94e5008 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:36.805987 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c6216-1ea4-4b29-8803-29e7e94e5008 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:36.838848 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ed643-4e95-4bb6-aece-f75e40acd04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:36.839444 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ed643-4e95-4bb6-aece-f75e40acd04a None admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:36.839724 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ed643-4e95-4bb6-aece-f75e40acd04a None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:36.839994 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ed643-4e95-4bb6-aece-f75e40acd04a None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:36.840639 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ed643-4e95-4bb6-aece-f75e40acd04a None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:36.842014 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ed643-4e95-4bb6-aece-f75e40acd04a None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:36.858277 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 589/1169] 10.222.0.76 () {60 vars in 1138 bytes} [Wed Aug 10 22:36:36 2022] POST /identity/v3/domains => generated 267 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:36:38.905666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ce8a9-5e30-4f89-b024-c4de17a74e93 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:38.906257 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ce8a9-5e30-4f89-b024-c4de17a74e93 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:38.906257 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ce8a9-5e30-4f89-b024-c4de17a74e93 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:38.907548 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 581/1170] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:36:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:36:38.913249 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9a3b5-2e4f-4eb0-9788-c04a3c2a3adc None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:38.913676 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9a3b5-2e4f-4eb0-9788-c04a3c2a3adc None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:38.913834 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9a3b5-2e4f-4eb0-9788-c04a3c2a3adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:38.927654 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:38.933512 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-87d9a3b5-2e4f-4eb0-9788-c04a3c2a3adc None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:36:38.934958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-87d9a3b5-2e4f-4eb0-9788-c04a3c2a3adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:36:39.000931 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 590/1171] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:36:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:36:39.024723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ce8a9-5e30-4f89-b024-c4de17a74e93 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:39.124003 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ce8a9-5e30-4f89-b024-c4de17a74e93 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:39.147322 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:39.157761 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef645a-8856-457a-8e95-aec44ccbf94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:39.158323 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef645a-8856-457a-8e95-aec44ccbf94a None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:39.158598 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef645a-8856-457a-8e95-aec44ccbf94a None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:39.158891 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef645a-8856-457a-8e95-aec44ccbf94a None admin] PATH_INFO: `/v3/domains/27b8699c63e044fb8e0bb22c349fd040` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:39.168799 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aef645a-8856-457a-8e95-aec44ccbf94a None admin] RBAC: Authorizing `identity:get_domain(domain_id=27b8699c63e044fb8e0bb22c349fd040)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:39.170265 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aef645a-8856-457a-8e95-aec44ccbf94a None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:39.171831 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 582/1172] 10.222.0.76 () {56 vars in 1183 bytes} [Wed Aug 10 22:36:39 2022] GET /identity/v3/domains/27b8699c63e044fb8e0bb22c349fd040 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:39.178271 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9a3b5-2e4f-4eb0-9788-c04a3c2a3adc None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:39.275216 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9a3b5-2e4f-4eb0-9788-c04a3c2a3adc None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:39.308515 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ad9d9-946d-49fa-9bf0-c01785f5e738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:39.309125 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ad9d9-946d-49fa-9bf0-c01785f5e738 None admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:39.309503 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ad9d9-946d-49fa-9bf0-c01785f5e738 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:39.309780 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ad9d9-946d-49fa-9bf0-c01785f5e738 None admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:39.310665 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ad9d9-946d-49fa-9bf0-c01785f5e738 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:39.312099 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ad9d9-946d-49fa-9bf0-c01785f5e738 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:39.317393 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:39.338480 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 591/1173] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 10 22:36:39 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:36:41.439042 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbcd569-f7be-4354-b7ef-954494dffa88 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:41.439458 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbcd569-f7be-4354-b7ef-954494dffa88 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:41.439599 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbcd569-f7be-4354-b7ef-954494dffa88 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:41.441630 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 583/1174] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:36:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:36:41.447044 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d139d-35bb-4d9a-8c44-76307c7ea083 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:41.447483 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d139d-35bb-4d9a-8c44-76307c7ea083 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:41.447575 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d139d-35bb-4d9a-8c44-76307c7ea083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:41.466068 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-f31d139d-35bb-4d9a-8c44-76307c7ea083 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:36:41.467252 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-f31d139d-35bb-4d9a-8c44-76307c7ea083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:36:41.531869 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 592/1175] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:36:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:36:41.555845 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbcd569-f7be-4354-b7ef-954494dffa88 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:41.655724 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbcd569-f7be-4354-b7ef-954494dffa88 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:41.688292 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92976fa6-441a-4ff6-a5fb-5fa9ccbd073a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:41.688849 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92976fa6-441a-4ff6-a5fb-5fa9ccbd073a None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:41.689129 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92976fa6-441a-4ff6-a5fb-5fa9ccbd073a None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:41.689434 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92976fa6-441a-4ff6-a5fb-5fa9ccbd073a None admin] PATH_INFO: `/v3/domains/27b8699c63e044fb8e0bb22c349fd040` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:41.690530 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92976fa6-441a-4ff6-a5fb-5fa9ccbd073a None admin] RBAC: Authorizing `identity:get_domain(domain_id=27b8699c63e044fb8e0bb22c349fd040)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:41.692592 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92976fa6-441a-4ff6-a5fb-5fa9ccbd073a None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:41.694451 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 584/1176] 10.222.0.76 () {56 vars in 1183 bytes} [Wed Aug 10 22:36:41 2022] GET /identity/v3/domains/27b8699c63e044fb8e0bb22c349fd040 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:41.700216 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d139d-35bb-4d9a-8c44-76307c7ea083 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:41.805667 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d139d-35bb-4d9a-8c44-76307c7ea083 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:41.823841 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:41.843371 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c1c1d1-6634-4d43-8f54-e18c4d61ade5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:41.843980 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1c1d1-6634-4d43-8f54-e18c4d61ade5 None admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:41.844256 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1c1d1-6634-4d43-8f54-e18c4d61ade5 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:41.844524 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1c1d1-6634-4d43-8f54-e18c4d61ade5 None admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:41.845168 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c1c1d1-6634-4d43-8f54-e18c4d61ade5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:41.846719 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c1c1d1-6634-4d43-8f54-e18c4d61ade5 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:41.872858 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 593/1177] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 10 22:36:41 2022] POST /identity/v3/users => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:36:43.938541 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ade532-577a-4bb8-bcc4-f9f5ca08e64b None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:43.939155 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ade532-577a-4bb8-bcc4-f9f5ca08e64b None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:43.939155 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ade532-577a-4bb8-bcc4-f9f5ca08e64b None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:43.941035 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 585/1178] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:36:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:36:43.946447 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9639a3-e3a6-4a92-b604-ea014ef1c2e3 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:43.946883 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9639a3-e3a6-4a92-b604-ea014ef1c2e3 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:43.946981 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9639a3-e3a6-4a92-b604-ea014ef1c2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:43.965653 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-0f9639a3-e3a6-4a92-b604-ea014ef1c2e3 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:36:43.966850 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-0f9639a3-e3a6-4a92-b604-ea014ef1c2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:36:44.032293 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 594/1179] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:36:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:36:44.055926 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ade532-577a-4bb8-bcc4-f9f5ca08e64b None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:44.111442 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:44.161601 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ade532-577a-4bb8-bcc4-f9f5ca08e64b None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:44.195861 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43524edd-1b3f-42d1-8fbf-98a4993f25a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:44.196426 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43524edd-1b3f-42d1-8fbf-98a4993f25a3 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:44.196705 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43524edd-1b3f-42d1-8fbf-98a4993f25a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:44.196978 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43524edd-1b3f-42d1-8fbf-98a4993f25a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:44.207644 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43524edd-1b3f-42d1-8fbf-98a4993f25a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:36:44.212197 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-43524edd-1b3f-42d1-8fbf-98a4993f25a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:36:44.212197 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:36:44.212197 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:36:44.212197 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:36:44.212197 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:36:44.212197 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:36:44.212197 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:36:44.212197 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:36:44.212197 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:36:44.212197 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:36:44.212197 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:36:44.212197 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:36:44.212197 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:36:44.212197 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:36:44.212197 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:36:44.212197 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:36:44.213443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:36:44.213443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:36:44.213443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:36:44.213443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:36:44.213443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:36:44.213443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:36:44.213443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:36:44.213443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:36:44.213443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:36:44.213443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:36:44.213443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:36:44.213443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:36:44.213443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:36:44.213443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:36:44.213443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:36:44.213443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:36:44.214593 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:36:44.214593 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:36:44.214593 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:36:44.214593 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:36:44.214593 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:36:44.214593 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:36:44.214593 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:36:44.214593 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:36:44.214593 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:36:44.214593 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:36:44.214593 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:36:44.214593 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:36:44.214593 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 586/1180] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:36:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:36:44.219637 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9639a3-e3a6-4a92-b604-ea014ef1c2e3 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:44.317106 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9639a3-e3a6-4a92-b604-ea014ef1c2e3 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:44.319401 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:44.351118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442875e1-5ae4-43f4-9360-95b5c3762308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:44.351717 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442875e1-5ae4-43f4-9360-95b5c3762308 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:44.351994 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442875e1-5ae4-43f4-9360-95b5c3762308 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:44.352264 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442875e1-5ae4-43f4-9360-95b5c3762308 None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:44.352918 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442875e1-5ae4-43f4-9360-95b5c3762308 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:36:44.353215 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442875e1-5ae4-43f4-9360-95b5c3762308 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:44.354605 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442875e1-5ae4-43f4-9360-95b5c3762308 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:44.363139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 595/1181] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:36:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:36:44.368967 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43524edd-1b3f-42d1-8fbf-98a4993f25a3 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:44.469807 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43524edd-1b3f-42d1-8fbf-98a4993f25a3 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:44.482155 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:44.503887 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e370a-aa7b-42ea-b5d8-ea1b9fed6eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:44.504510 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e370a-aa7b-42ea-b5d8-ea1b9fed6eeb None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:44.504787 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e370a-aa7b-42ea-b5d8-ea1b9fed6eeb None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:44.505059 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e370a-aa7b-42ea-b5d8-ea1b9fed6eeb None admin] PATH_INFO: `/v3/users/7833a33a972d48ddb42365fdf64ab239` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:44.517947 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e370a-aa7b-42ea-b5d8-ea1b9fed6eeb None admin] RBAC: Authorizing `identity:get_user(user_id=7833a33a972d48ddb42365fdf64ab239)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:44.520055 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e370a-aa7b-42ea-b5d8-ea1b9fed6eeb None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:44.521674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 587/1182] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:36:44 2022] GET /identity/v3/users/7833a33a972d48ddb42365fdf64ab239 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:44.527519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442875e1-5ae4-43f4-9360-95b5c3762308 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:44.623108 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442875e1-5ae4-43f4-9360-95b5c3762308 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:44.656344 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54a16d-9d11-4586-a2f9-f0cf4f9ea214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:44.656938 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54a16d-9d11-4586-a2f9-f0cf4f9ea214 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:44.657217 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54a16d-9d11-4586-a2f9-f0cf4f9ea214 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:44.657517 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54a16d-9d11-4586-a2f9-f0cf4f9ea214 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:44.689857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:44.724432 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab54a16d-9d11-4586-a2f9-f0cf4f9ea214 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:44.725855 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab54a16d-9d11-4586-a2f9-f0cf4f9ea214 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:44.820613 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 596/1183] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:36:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 22:36:44.826552 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e370a-aa7b-42ea-b5d8-ea1b9fed6eeb None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:44.922839 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e370a-aa7b-42ea-b5d8-ea1b9fed6eeb None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:44.957679 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912447f7-7675-4948-8ab1-901627ae865b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:44.958298 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912447f7-7675-4948-8ab1-901627ae865b None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:44.958578 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912447f7-7675-4948-8ab1-901627ae865b None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:44.958852 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912447f7-7675-4948-8ab1-901627ae865b None admin] PATH_INFO: `/v3/projects/e1832bca14f640789655bccda8dff876` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:44.959960 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912447f7-7675-4948-8ab1-901627ae865b None admin] RBAC: Authorizing `identity:get_project(project_id=e1832bca14f640789655bccda8dff876)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:44.961429 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912447f7-7675-4948-8ab1-901627ae865b None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:44.963052 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 588/1184] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:36:44 2022] GET /identity/v3/projects/e1832bca14f640789655bccda8dff876 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:44.969319 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54a16d-9d11-4586-a2f9-f0cf4f9ea214 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:45.033360 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:45.068463 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54a16d-9d11-4586-a2f9-f0cf4f9ea214 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:45.100835 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6373d0f2-89ca-4866-bb06-9da58186acf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:45.101458 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373d0f2-89ca-4866-bb06-9da58186acf2 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:45.101741 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373d0f2-89ca-4866-bb06-9da58186acf2 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:45.102015 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373d0f2-89ca-4866-bb06-9da58186acf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:45.102732 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6373d0f2-89ca-4866-bb06-9da58186acf2 None admin] RBAC: Adding query filter params (role.id=6641e95fdc1f4c098ebc372bb692d227, scope.project.id=e1832bca14f640789655bccda8dff876, user.id=7833a33a972d48ddb42365fdf64ab239) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:36:45.103037 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6373d0f2-89ca-4866-bb06-9da58186acf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:45.104507 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6373d0f2-89ca-4866-bb06-9da58186acf2 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:45.109816 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 597/1185] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:36:44 2022] GET /identity/v3/role_assignments?user.id=7833a33a972d48ddb42365fdf64ab239&scope.project.id=e1832bca14f640789655bccda8dff876&role.id=6641e95fdc1f4c098ebc372bb692d227 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:47.173723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14692c4-d4c1-4242-819f-a613ffca7648 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:47.174795 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14692c4-d4c1-4242-819f-a613ffca7648 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:47.174795 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14692c4-d4c1-4242-819f-a613ffca7648 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:47.176186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 589/1186] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:36:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:36:47.181638 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652227a7-8fdc-4f1d-9669-8a3bfa9dc33e None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:47.182107 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652227a7-8fdc-4f1d-9669-8a3bfa9dc33e None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:47.182209 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652227a7-8fdc-4f1d-9669-8a3bfa9dc33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:47.202496 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-652227a7-8fdc-4f1d-9669-8a3bfa9dc33e None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:36:47.203960 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-652227a7-8fdc-4f1d-9669-8a3bfa9dc33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:36:47.287122 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 598/1187] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:36:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:36:47.308545 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14692c4-d4c1-4242-819f-a613ffca7648 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:47.349658 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:47.409753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14692c4-d4c1-4242-819f-a613ffca7648 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:47.445346 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e240cf-ffca-4ee5-9005-0fb8eeacf609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:47.445975 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e240cf-ffca-4ee5-9005-0fb8eeacf609 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:47.446251 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e240cf-ffca-4ee5-9005-0fb8eeacf609 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:47.446521 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e240cf-ffca-4ee5-9005-0fb8eeacf609 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:47.457863 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e240cf-ffca-4ee5-9005-0fb8eeacf609 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:36:47.462457 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-68e240cf-ffca-4ee5-9005-0fb8eeacf609 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:36:47.462457 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:36:47.462457 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:36:47.462457 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:36:47.462457 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:36:47.462457 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:36:47.462457 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:36:47.462457 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:36:47.462457 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:36:47.462457 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:36:47.462457 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:36:47.462457 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:36:47.462457 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:36:47.462457 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:36:47.462457 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:36:47.462457 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:36:47.463524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:36:47.463524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:36:47.463524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:36:47.463524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:36:47.463524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:36:47.463524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:36:47.463524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:36:47.463524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:36:47.463524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:36:47.463524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:36:47.463524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:36:47.463524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:36:47.463524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:36:47.463524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:36:47.463524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:36:47.463524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:36:47.464422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:36:47.464422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:36:47.464422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:36:47.464422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:36:47.464422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:36:47.464422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:36:47.464422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:36:47.464422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:36:47.464422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:36:47.464422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:36:47.464422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:36:47.464422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:36:47.464422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 590/1188] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:36:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:36:47.469699 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652227a7-8fdc-4f1d-9669-8a3bfa9dc33e None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:47.538743 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:47.590769 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652227a7-8fdc-4f1d-9669-8a3bfa9dc33e None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:47.625782 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374401f-a1f1-4968-951e-b897399c2799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:47.626390 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374401f-a1f1-4968-951e-b897399c2799 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:47.626671 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374401f-a1f1-4968-951e-b897399c2799 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:47.627479 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374401f-a1f1-4968-951e-b897399c2799 None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:47.628289 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374401f-a1f1-4968-951e-b897399c2799 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:36:47.628588 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374401f-a1f1-4968-951e-b897399c2799 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:47.629987 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374401f-a1f1-4968-951e-b897399c2799 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:47.638016 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 599/1189] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:36:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:47.644920 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e240cf-ffca-4ee5-9005-0fb8eeacf609 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:47.738790 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:47.742773 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e240cf-ffca-4ee5-9005-0fb8eeacf609 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:47.776812 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e0f0f-0ecf-4cdd-b34d-f42e9948c480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:47.777474 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e0f0f-0ecf-4cdd-b34d-f42e9948c480 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:47.777754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e0f0f-0ecf-4cdd-b34d-f42e9948c480 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:47.778025 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e0f0f-0ecf-4cdd-b34d-f42e9948c480 None admin] PATH_INFO: `/v3/users/7833a33a972d48ddb42365fdf64ab239` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:47.779136 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093e0f0f-0ecf-4cdd-b34d-f42e9948c480 None admin] RBAC: Authorizing `identity:get_user(user_id=7833a33a972d48ddb42365fdf64ab239)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:47.780564 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093e0f0f-0ecf-4cdd-b34d-f42e9948c480 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:47.782167 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 591/1190] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:36:47 2022] GET /identity/v3/users/7833a33a972d48ddb42365fdf64ab239 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:47.788017 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374401f-a1f1-4968-951e-b897399c2799 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:47.888034 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374401f-a1f1-4968-951e-b897399c2799 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:47.910411 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:47.925854 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc5952f-a4fa-4f6b-ac87-e12273f4e630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:47.926489 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5952f-a4fa-4f6b-ac87-e12273f4e630 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:47.926771 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5952f-a4fa-4f6b-ac87-e12273f4e630 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:47.927044 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5952f-a4fa-4f6b-ac87-e12273f4e630 None admin] PATH_INFO: `/v3/projects/e1832bca14f640789655bccda8dff876` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:47.928193 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc5952f-a4fa-4f6b-ac87-e12273f4e630 None admin] RBAC: Authorizing `identity:get_project(project_id=e1832bca14f640789655bccda8dff876)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:47.929688 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc5952f-a4fa-4f6b-ac87-e12273f4e630 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:47.931324 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 600/1191] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:36:47 2022] GET /identity/v3/projects/e1832bca14f640789655bccda8dff876 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:47.937556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e0f0f-0ecf-4cdd-b34d-f42e9948c480 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:48.036600 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e0f0f-0ecf-4cdd-b34d-f42e9948c480 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:48.070972 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef0d7bd-f125-46cf-9042-93bf1c9bb92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:48.071619 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0d7bd-f125-46cf-9042-93bf1c9bb92c None admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:48.071900 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0d7bd-f125-46cf-9042-93bf1c9bb92c None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:48.072168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0d7bd-f125-46cf-9042-93bf1c9bb92c None admin] PATH_INFO: `/v3/projects/e1832bca14f640789655bccda8dff876/users/7833a33a972d48ddb42365fdf64ab239/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:48.074061 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef0d7bd-f125-46cf-9042-93bf1c9bb92c None admin] RBAC: Authorizing `identity:create_grant(project_id=e1832bca14f640789655bccda8dff876, user_id=7833a33a972d48ddb42365fdf64ab239, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:48.075484 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef0d7bd-f125-46cf-9042-93bf1c9bb92c None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:48.087864 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 592/1192] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 10 22:36:47 2022] PUT /identity/v3/projects/e1832bca14f640789655bccda8dff876/users/7833a33a972d48ddb42365fdf64ab239/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:36:50.067370 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3fbc19-d383-4fa5-9261-bbfe31683421 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:50.067907 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3fbc19-d383-4fa5-9261-bbfe31683421 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:50.067907 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3fbc19-d383-4fa5-9261-bbfe31683421 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:50.069316 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 601/1193] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:36:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:36:50.074516 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b333bba-a891-41fd-9fe7-5375e206fcf5 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:50.074916 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b333bba-a891-41fd-9fe7-5375e206fcf5 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:50.075060 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b333bba-a891-41fd-9fe7-5375e206fcf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:50.093239 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-8b333bba-a891-41fd-9fe7-5375e206fcf5 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:36:50.094449 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-8b333bba-a891-41fd-9fe7-5375e206fcf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:36:50.115328 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:50.163001 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 593/1194] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:36:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:36:50.186441 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3fbc19-d383-4fa5-9261-bbfe31683421 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:50.289208 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3fbc19-d383-4fa5-9261-bbfe31683421 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:50.326329 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef98a12-70b0-412e-b3b6-99acb65bd4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:50.326802 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef98a12-70b0-412e-b3b6-99acb65bd4bc None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:50.327086 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef98a12-70b0-412e-b3b6-99acb65bd4bc None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:50.327360 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef98a12-70b0-412e-b3b6-99acb65bd4bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:50.337870 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef98a12-70b0-412e-b3b6-99acb65bd4bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:36:50.342510 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-4ef98a12-70b0-412e-b3b6-99acb65bd4bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:36:50.342510 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:36:50.342510 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:36:50.342510 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:36:50.342510 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:36:50.342510 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:36:50.342510 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:36:50.342510 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:36:50.342510 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:36:50.342510 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:36:50.342510 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:36:50.342510 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:36:50.342510 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:36:50.342510 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:36:50.342510 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:36:50.342510 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:36:50.343521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:36:50.343521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:36:50.343521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:36:50.343521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:36:50.343521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:36:50.343521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:36:50.343521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:36:50.343521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:36:50.343521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:36:50.343521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:36:50.343521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:36:50.343521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:36:50.343521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:36:50.343521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:36:50.343521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:36:50.343521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:36:50.344357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:36:50.344357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:36:50.344357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:36:50.344357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:36:50.344357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:36:50.344357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:36:50.344357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:36:50.344357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:36:50.344357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:36:50.344357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:36:50.344357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:36:50.344357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:36:50.344357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 602/1195] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:36:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:36:50.350467 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b333bba-a891-41fd-9fe7-5375e206fcf5 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:50.448988 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b333bba-a891-41fd-9fe7-5375e206fcf5 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:50.483202 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9e4ad-c8ab-4a0c-ac42-970f572434f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:50.483816 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9e4ad-c8ab-4a0c-ac42-970f572434f6 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:50.484099 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9e4ad-c8ab-4a0c-ac42-970f572434f6 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:50.484376 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9e4ad-c8ab-4a0c-ac42-970f572434f6 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:50.485034 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf9e4ad-c8ab-4a0c-ac42-970f572434f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:36:50.485365 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf9e4ad-c8ab-4a0c-ac42-970f572434f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:50.488206 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf9e4ad-c8ab-4a0c-ac42-970f572434f6 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:50.496746 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 594/1196] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:36:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:50.503075 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef98a12-70b0-412e-b3b6-99acb65bd4bc None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:50.555007 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:50.605360 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef98a12-70b0-412e-b3b6-99acb65bd4bc None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:50.638635 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb3ca0b-9454-453f-b313-4f9a86f563e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:50.639248 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3ca0b-9454-453f-b313-4f9a86f563e5 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:50.639532 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3ca0b-9454-453f-b313-4f9a86f563e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:50.639807 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3ca0b-9454-453f-b313-4f9a86f563e5 None admin] PATH_INFO: `/v3/users/7833a33a972d48ddb42365fdf64ab239` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:50.640943 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb3ca0b-9454-453f-b313-4f9a86f563e5 None admin] RBAC: Authorizing `identity:get_user(user_id=7833a33a972d48ddb42365fdf64ab239)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:50.643378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb3ca0b-9454-453f-b313-4f9a86f563e5 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:50.644969 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 603/1197] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 10 22:36:50 2022] GET /identity/v3/users/7833a33a972d48ddb42365fdf64ab239 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:50.650905 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9e4ad-c8ab-4a0c-ac42-970f572434f6 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:50.670609 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:50.749978 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9e4ad-c8ab-4a0c-ac42-970f572434f6 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:50.783320 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9fef9f-2207-492f-b7e8-1f614063caa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:50.783931 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9fef9f-2207-492f-b7e8-1f614063caa6 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:50.784207 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9fef9f-2207-492f-b7e8-1f614063caa6 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:50.784479 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9fef9f-2207-492f-b7e8-1f614063caa6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:50.850610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9fef9f-2207-492f-b7e8-1f614063caa6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:50.852043 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9fef9f-2207-492f-b7e8-1f614063caa6 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:50.868772 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:50.948394 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 595/1198] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:36:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 22:36:50.955547 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb3ca0b-9454-453f-b313-4f9a86f563e5 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:51.053593 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb3ca0b-9454-453f-b313-4f9a86f563e5 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:51.070852 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:51.087834 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e2bcc-15a0-4d8e-bdf0-bfe8a16bb4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:51.088466 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e2bcc-15a0-4d8e-bdf0-bfe8a16bb4e5 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:51.088752 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e2bcc-15a0-4d8e-bdf0-bfe8a16bb4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:51.089033 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e2bcc-15a0-4d8e-bdf0-bfe8a16bb4e5 None admin] PATH_INFO: `/v3/projects/e1832bca14f640789655bccda8dff876` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:51.090191 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4e2bcc-15a0-4d8e-bdf0-bfe8a16bb4e5 None admin] RBAC: Authorizing `identity:get_project(project_id=e1832bca14f640789655bccda8dff876)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:51.091633 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4e2bcc-15a0-4d8e-bdf0-bfe8a16bb4e5 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:51.093257 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 604/1199] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 10 22:36:50 2022] GET /identity/v3/projects/e1832bca14f640789655bccda8dff876 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:36:51.099497 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9fef9f-2207-492f-b7e8-1f614063caa6 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:36:51.197317 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9fef9f-2207-492f-b7e8-1f614063caa6 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:36:51.214646 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:36:51.232905 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed34a88-d446-42a5-b87d-7bdab96a9ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:36:51.233537 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed34a88-d446-42a5-b87d-7bdab96a9ad0 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:36:51.233812 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed34a88-d446-42a5-b87d-7bdab96a9ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:36:51.234075 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed34a88-d446-42a5-b87d-7bdab96a9ad0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:36:51.234783 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed34a88-d446-42a5-b87d-7bdab96a9ad0 None admin] RBAC: Adding query filter params (role.id=6641e95fdc1f4c098ebc372bb692d227, scope.project.id=e1832bca14f640789655bccda8dff876, user.id=7833a33a972d48ddb42365fdf64ab239) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:36:51.235083 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed34a88-d446-42a5-b87d-7bdab96a9ad0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:36:51.237120 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed34a88-d446-42a5-b87d-7bdab96a9ad0 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:36:51.242787 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 596/1200] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:36:51 2022] GET /identity/v3/role_assignments?user.id=7833a33a972d48ddb42365fdf64ab239&scope.project.id=e1832bca14f640789655bccda8dff876&role.id=6641e95fdc1f4c098ebc372bb692d227 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:37:01.083692 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=7 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:37:01.240923 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=9 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:37:47.536976 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316fdd59-109a-4ca2-bdc9-18bade7febd8 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:37:47.537596 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316fdd59-109a-4ca2-bdc9-18bade7febd8 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:37:47.537596 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316fdd59-109a-4ca2-bdc9-18bade7febd8 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:37:47.539092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 605/1201] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:37:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:37:47.545006 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a70a2-2334-4e06-8a06-a62a19ae72e9 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:37:47.545576 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a70a2-2334-4e06-8a06-a62a19ae72e9 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:37:47.545576 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a70a2-2334-4e06-8a06-a62a19ae72e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:37:47.568933 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-a68a70a2-2334-4e06-8a06-a62a19ae72e9 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:37:47.570309 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-a68a70a2-2334-4e06-8a06-a62a19ae72e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:37:47.683634 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 597/1202] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:37:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:37:47.713837 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316fdd59-109a-4ca2-bdc9-18bade7febd8 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:37:47.873517 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316fdd59-109a-4ca2-bdc9-18bade7febd8 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:37:47.926238 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d902f7c-4ab7-4c36-874c-614b7bbca588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:37:47.926894 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d902f7c-4ab7-4c36-874c-614b7bbca588 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:37:47.927176 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d902f7c-4ab7-4c36-874c-614b7bbca588 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:37:47.927450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d902f7c-4ab7-4c36-874c-614b7bbca588 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:37:47.939047 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d902f7c-4ab7-4c36-874c-614b7bbca588 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:37:47.943640 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d902f7c-4ab7-4c36-874c-614b7bbca588 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:37:47.966099 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-1d902f7c-4ab7-4c36-874c-614b7bbca588 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:37:47.966099 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:37:47.966099 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:37:47.966099 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:37:47.966099 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:37:47.966099 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:37:47.966099 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:37:47.966099 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:37:47.966099 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:37:47.966099 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:37:47.966099 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:37:47.966099 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:37:47.966099 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:37:47.966099 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:37:47.966099 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:37:47.966099 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:37:47.967722 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:37:47.967722 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:37:47.967722 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:37:47.967722 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:37:47.967722 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:37:47.967722 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:37:47.967722 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:37:47.967722 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:37:47.967722 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:37:47.967722 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:37:47.967722 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:37:47.967722 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:37:47.967722 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:37:47.967722 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:37:47.967722 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:37:47.967722 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:37:47.967722 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:37:47.968982 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:37:47.968982 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:37:47.968982 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:37:47.968982 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 606/1203] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:37:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:37:47.974529 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a70a2-2334-4e06-8a06-a62a19ae72e9 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:37:48.133451 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a70a2-2334-4e06-8a06-a62a19ae72e9 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:37:48.205416 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:37:48.213121 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa5cf4-74ce-4516-932a-25fd11a5ef86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:37:48.214088 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa5cf4-74ce-4516-932a-25fd11a5ef86 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:37:48.214677 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa5cf4-74ce-4516-932a-25fd11a5ef86 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:37:48.215005 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa5cf4-74ce-4516-932a-25fd11a5ef86 None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:37:48.215859 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaa5cf4-74ce-4516-932a-25fd11a5ef86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:37:48.216420 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaa5cf4-74ce-4516-932a-25fd11a5ef86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:37:48.218322 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaa5cf4-74ce-4516-932a-25fd11a5ef86 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:37:48.239544 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 598/1204] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:37:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:37:48.242312 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d902f7c-4ab7-4c36-874c-614b7bbca588 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:37:48.316566 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:37:48.382198 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d902f7c-4ab7-4c36-874c-614b7bbca588 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:37:48.446911 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af684afb-f7f8-4c90-a7e9-1c3a8ef7eeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:37:48.447544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af684afb-f7f8-4c90-a7e9-1c3a8ef7eeb4 None admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:37:48.447820 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af684afb-f7f8-4c90-a7e9-1c3a8ef7eeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:37:48.448126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af684afb-f7f8-4c90-a7e9-1c3a8ef7eeb4 None admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:37:48.448838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af684afb-f7f8-4c90-a7e9-1c3a8ef7eeb4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:37:48.450393 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af684afb-f7f8-4c90-a7e9-1c3a8ef7eeb4 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:37:48.482554 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 607/1205] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 10 22:37:48 2022] POST /identity/v3/users => generated 251 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:37:50.799622 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9fbb4-c2f5-41e3-99f0-381829d04fc6 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:37:50.799622 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9fbb4-c2f5-41e3-99f0-381829d04fc6 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:37:50.799622 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9fbb4-c2f5-41e3-99f0-381829d04fc6 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:37:50.799622 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 599/1206] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:37:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:37:50.805956 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70253119-7317-4266-9d75-b7dc71018842 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:37:50.806395 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70253119-7317-4266-9d75-b7dc71018842 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:37:50.806616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70253119-7317-4266-9d75-b7dc71018842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:37:50.827600 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-70253119-7317-4266-9d75-b7dc71018842 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:37:50.829481 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-70253119-7317-4266-9d75-b7dc71018842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:37:50.908613 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 608/1207] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:37:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:37:50.933198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9fbb4-c2f5-41e3-99f0-381829d04fc6 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:37:51.071056 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9fbb4-c2f5-41e3-99f0-381829d04fc6 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:37:51.115179 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a181ce-1017-485f-bbf0-402f9b6248a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:37:51.115811 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a181ce-1017-485f-bbf0-402f9b6248a3 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:37:51.116085 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a181ce-1017-485f-bbf0-402f9b6248a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:37:51.116349 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a181ce-1017-485f-bbf0-402f9b6248a3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:37:51.130160 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a181ce-1017-485f-bbf0-402f9b6248a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:37:51.135437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-a8a181ce-1017-485f-bbf0-402f9b6248a3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:37:51.135437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:37:51.135437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:37:51.135437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:37:51.135437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:37:51.135437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:37:51.135437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:37:51.135437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:37:51.135437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:37:51.135437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:37:51.135437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:37:51.135437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:37:51.135437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:37:51.135437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:37:51.135437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:37:51.135437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:37:51.136460 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:37:51.136460 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:37:51.136460 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:37:51.136460 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:37:51.136460 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:37:51.136460 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:37:51.136460 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:37:51.136460 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:37:51.136460 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:37:51.136460 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:37:51.136460 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:37:51.136460 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:37:51.136460 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:37:51.136460 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:37:51.136460 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:37:51.136460 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:37:51.137341 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:37:51.137341 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:37:51.137341 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:37:51.137341 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:37:51.137341 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:37:51.137341 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:37:51.137341 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:37:51.137341 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:37:51.137341 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:37:51.137341 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:37:51.137341 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:37:51.137341 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:37:51.142460 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 600/1208] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 10 22:37:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:37:51.143455 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70253119-7317-4266-9d75-b7dc71018842 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:37:51.191375 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:37:51.267286 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70253119-7317-4266-9d75-b7dc71018842 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:37:51.311444 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b0e05-c3c6-4b6c-b8ff-e188a5c81309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:37:51.312080 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b0e05-c3c6-4b6c-b8ff-e188a5c81309 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:37:51.312366 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b0e05-c3c6-4b6c-b8ff-e188a5c81309 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:37:51.313522 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b0e05-c3c6-4b6c-b8ff-e188a5c81309 None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:37:51.314338 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b0e05-c3c6-4b6c-b8ff-e188a5c81309 None admin] RBAC: Adding query filter params (name=service) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:37:51.314610 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b0e05-c3c6-4b6c-b8ff-e188a5c81309 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:37:51.316151 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b0e05-c3c6-4b6c-b8ff-e188a5c81309 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:37:51.329130 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 609/1209] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 10 22:37:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:37:51.335232 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a181ce-1017-485f-bbf0-402f9b6248a3 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:37:51.399604 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:37:51.467303 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a181ce-1017-485f-bbf0-402f9b6248a3 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:37:51.510529 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb77fa1-882f-4581-ab09-7f1725c3f05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:37:51.511175 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb77fa1-882f-4581-ab09-7f1725c3f05f None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:37:51.511451 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb77fa1-882f-4581-ab09-7f1725c3f05f None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:37:51.511715 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb77fa1-882f-4581-ab09-7f1725c3f05f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:37:51.535833 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb77fa1-882f-4581-ab09-7f1725c3f05f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:37:51.537413 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb77fa1-882f-4581-ab09-7f1725c3f05f None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:37:51.554559 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-4cb77fa1-882f-4581-ab09-7f1725c3f05f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:37:51.554559 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:37:51.554559 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:37:51.554559 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:37:51.554559 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:37:51.554559 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:37:51.554559 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:37:51.554559 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:37:51.554559 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:37:51.554559 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:37:51.554559 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:37:51.554559 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:37:51.554559 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:37:51.554559 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:37:51.554559 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:37:51.554559 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:37:51.556422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:37:51.556422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:37:51.556422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:37:51.556422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:37:51.556422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:37:51.556422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:37:51.556422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:37:51.556422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:37:51.556422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:37:51.556422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:37:51.556422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:37:51.556422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:37:51.556422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:37:51.556422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:37:51.556422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:37:51.556422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:37:51.556422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:37:51.557602 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:37:51.557602 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:37:51.557602 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:37:51.557602 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 601/1210] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:37:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:37:51.561675 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b0e05-c3c6-4b6c-b8ff-e188a5c81309 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:37:51.675512 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:37:51.679383 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b0e05-c3c6-4b6c-b8ff-e188a5c81309 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:37:51.726126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f85b7f1-a0ef-45b4-8f6d-84b0bd3ceade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:37:51.726738 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85b7f1-a0ef-45b4-8f6d-84b0bd3ceade None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:37:51.727073 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85b7f1-a0ef-45b4-8f6d-84b0bd3ceade None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:37:51.727348 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85b7f1-a0ef-45b4-8f6d-84b0bd3ceade None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:37:51.728024 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f85b7f1-a0ef-45b4-8f6d-84b0bd3ceade None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:37:51.728327 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f85b7f1-a0ef-45b4-8f6d-84b0bd3ceade None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:37:51.730005 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f85b7f1-a0ef-45b4-8f6d-84b0bd3ceade None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:37:51.742790 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 610/1211] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:37:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:37:51.749202 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb77fa1-882f-4581-ab09-7f1725c3f05f None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:37:51.866658 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb77fa1-882f-4581-ab09-7f1725c3f05f None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:37:51.868431 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:37:51.907577 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a67693-2f04-41e0-8f12-15a1aa5b524b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:37:51.908558 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a67693-2f04-41e0-8f12-15a1aa5b524b None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:37:51.908558 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a67693-2f04-41e0-8f12-15a1aa5b524b None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:37:51.908902 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a67693-2f04-41e0-8f12-15a1aa5b524b None admin] PATH_INFO: `/v3/users/placement` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:37:51.921469 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a67693-2f04-41e0-8f12-15a1aa5b524b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:37:51.923975 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a67693-2f04-41e0-8f12-15a1aa5b524b None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:37:51.937224 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-59a67693-2f04-41e0-8f12-15a1aa5b524b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 22:37:51.937224 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:37:51.937224 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:37:51.937224 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:37:51.937224 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:37:51.937224 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:37:51.937224 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:37:51.937224 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:37:51.937224 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:37:51.937224 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:37:51.937224 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:37:51.937224 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:37:51.937224 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:37:51.937224 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:37:51.937224 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:37:51.937224 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:37:51.938849 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:37:51.938849 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:37:51.938849 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:37:51.938849 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:37:51.938849 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:37:51.938849 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:37:51.938849 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:37:51.938849 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:37:51.938849 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:37:51.938849 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:37:51.938849 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:37:51.938849 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:37:51.938849 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:37:51.938849 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:37:51.938849 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:37:51.938849 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:37:51.938849 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:37:51.938849 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:37:51.939935 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:37:51.939935 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:37:51.939935 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:37:51.939935 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:37:51.939935 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:37:51.939935 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:37:51.939935 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:37:51.939935 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:37:51.939935 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 22:37:51.939935 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:37:51.939935 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 602/1212] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:37:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:37:51.947157 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f85b7f1-a0ef-45b4-8f6d-84b0bd3ceade None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:37:52.068971 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f85b7f1-a0ef-45b4-8f6d-84b0bd3ceade None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:37:52.107159 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdeeba7-6fc7-4b79-9d7d-4a85c5ed5373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:37:52.109890 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdeeba7-6fc7-4b79-9d7d-4a85c5ed5373 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:37:52.110263 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdeeba7-6fc7-4b79-9d7d-4a85c5ed5373 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:37:52.110596 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdeeba7-6fc7-4b79-9d7d-4a85c5ed5373 None admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:37:52.111359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdeeba7-6fc7-4b79-9d7d-4a85c5ed5373 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:37:52.111645 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdeeba7-6fc7-4b79-9d7d-4a85c5ed5373 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:37:52.113087 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdeeba7-6fc7-4b79-9d7d-4a85c5ed5373 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:37:52.131573 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 611/1213] 10.222.0.76 () {56 vars in 1143 bytes} [Wed Aug 10 22:37:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:37:52.137368 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a67693-2f04-41e0-8f12-15a1aa5b524b None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:37:52.256920 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a67693-2f04-41e0-8f12-15a1aa5b524b None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:37:52.296428 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55127fd1-bcf1-47e8-b35d-992c918ae3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:37:52.297985 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55127fd1-bcf1-47e8-b35d-992c918ae3fa None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:37:52.298277 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55127fd1-bcf1-47e8-b35d-992c918ae3fa None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:37:52.298562 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55127fd1-bcf1-47e8-b35d-992c918ae3fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:37:52.336673 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:37:52.383270 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55127fd1-bcf1-47e8-b35d-992c918ae3fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:37:52.384918 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55127fd1-bcf1-47e8-b35d-992c918ae3fa None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:37:52.504903 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 603/1214] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:37:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 22:37:52.511720 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdeeba7-6fc7-4b79-9d7d-4a85c5ed5373 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:37:52.533180 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:37:52.628782 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdeeba7-6fc7-4b79-9d7d-4a85c5ed5373 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:37:52.667836 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd10695-4190-4689-804f-40ed46d08bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:37:52.668553 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd10695-4190-4689-804f-40ed46d08bc0 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:37:52.668837 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd10695-4190-4689-804f-40ed46d08bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:37:52.669129 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd10695-4190-4689-804f-40ed46d08bc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:37:52.681034 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd10695-4190-4689-804f-40ed46d08bc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:37:52.682893 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd10695-4190-4689-804f-40ed46d08bc0 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:37:52.700925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-cfd10695-4190-4689-804f-40ed46d08bc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:37:52.700925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:37:52.700925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:37:52.700925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:37:52.700925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:37:52.700925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:37:52.700925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:37:52.700925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:37:52.700925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:37:52.700925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:37:52.700925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:37:52.700925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:37:52.700925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:37:52.700925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:37:52.700925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:37:52.700925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:37:52.702245 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:37:52.702245 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:37:52.702245 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:37:52.702245 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:37:52.702245 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:37:52.702245 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:37:52.702245 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:37:52.702245 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:37:52.702245 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:37:52.702245 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:37:52.702245 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:37:52.702245 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:37:52.702245 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:37:52.702245 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:37:52.702245 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:37:52.702245 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:37:52.702245 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:37:52.703914 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:37:52.703914 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:37:52.703914 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:37:52.707903 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 612/1215] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:37:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:37:52.709322 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55127fd1-bcf1-47e8-b35d-992c918ae3fa None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:37:52.783778 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:37:52.828599 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55127fd1-bcf1-47e8-b35d-992c918ae3fa None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:37:52.866844 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd604eed-d0d9-4774-bf00-9ebc909edce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:37:52.867538 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd604eed-d0d9-4774-bf00-9ebc909edce1 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:37:52.867806 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd604eed-d0d9-4774-bf00-9ebc909edce1 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:37:52.868023 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd604eed-d0d9-4774-bf00-9ebc909edce1 None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:37:52.868779 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd604eed-d0d9-4774-bf00-9ebc909edce1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:37:52.869072 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd604eed-d0d9-4774-bf00-9ebc909edce1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:37:52.870541 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd604eed-d0d9-4774-bf00-9ebc909edce1 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:37:52.883884 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 604/1216] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:37:52.889448 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd10695-4190-4689-804f-40ed46d08bc0 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:37:52.980892 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:37:53.017082 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd10695-4190-4689-804f-40ed46d08bc0 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:37:53.057866 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a74a5-18ac-47a9-84b7-5a374adef359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:37:53.058426 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a74a5-18ac-47a9-84b7-5a374adef359 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:37:53.058713 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a74a5-18ac-47a9-84b7-5a374adef359 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:37:53.058990 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a74a5-18ac-47a9-84b7-5a374adef359 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:37:53.066623 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a74a5-18ac-47a9-84b7-5a374adef359 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:37:53.068075 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a74a5-18ac-47a9-84b7-5a374adef359 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:37:53.080811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-e95a74a5-18ac-47a9-84b7-5a374adef359 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:37:53.080811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:37:53.080811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:37:53.080811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:37:53.080811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:37:53.080811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:37:53.080811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:37:53.080811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:37:53.080811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:37:53.080811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:37:53.080811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:37:53.080811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:37:53.080811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:37:53.080811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:37:53.080811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:37:53.082185 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:37:53.082185 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:37:53.082185 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:37:53.082185 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:37:53.082185 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:37:53.082185 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:37:53.082185 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:37:53.082185 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:37:53.082185 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:37:53.082185 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:37:53.082185 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:37:53.082185 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:37:53.082185 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:37:53.082185 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:37:53.082185 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:37:53.082185 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:37:53.082185 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:37:53.083125 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:37:53.083125 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:37:53.083125 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:37:53.083125 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:37:53.083125 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:37:53.083125 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:37:53.083125 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:37:53.083125 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:37:53.083125 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 613/1217] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:37:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:37:53.089778 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd604eed-d0d9-4774-bf00-9ebc909edce1 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:37:53.203195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd604eed-d0d9-4774-bf00-9ebc909edce1 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:37:53.227857 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:37:53.242560 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878de0dc-5313-4133-a05c-ceda0533baae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:37:53.243287 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878de0dc-5313-4133-a05c-ceda0533baae None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:37:53.243666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878de0dc-5313-4133-a05c-ceda0533baae None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:37:53.244066 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878de0dc-5313-4133-a05c-ceda0533baae None admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:37:53.244900 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878de0dc-5313-4133-a05c-ceda0533baae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:37:53.245307 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878de0dc-5313-4133-a05c-ceda0533baae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:37:53.247086 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878de0dc-5313-4133-a05c-ceda0533baae None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:37:53.265456 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 605/1218] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:37:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:37:53.267088 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a74a5-18ac-47a9-84b7-5a374adef359 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:37:53.385429 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a74a5-18ac-47a9-84b7-5a374adef359 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:37:53.419102 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:37:53.432158 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d46640e-9370-418b-b152-96e7d91fa8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:37:53.432889 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46640e-9370-418b-b152-96e7d91fa8f9 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:37:53.433252 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46640e-9370-418b-b152-96e7d91fa8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:37:53.433639 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46640e-9370-418b-b152-96e7d91fa8f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:37:53.434423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d46640e-9370-418b-b152-96e7d91fa8f9 None admin] RBAC: Adding query filter params (role.id=5dfd0f952cd641bbb2257860c4c892ac, scope.project.id=3caa2ea846f7449b8c483788fc5846c8, user.id=9f46dee0f73e49eab54c11556e3b86bf) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:37:53.434737 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d46640e-9370-418b-b152-96e7d91fa8f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:37:53.436479 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d46640e-9370-418b-b152-96e7d91fa8f9 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:37:53.443362 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 614/1219] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:37:53 2022] GET /identity/v3/role_assignments?user.id=9f46dee0f73e49eab54c11556e3b86bf&scope.project.id=3caa2ea846f7449b8c483788fc5846c8&role.id=5dfd0f952cd641bbb2257860c4c892ac => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:37:55.556342 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc0e17-566a-432e-a236-d4ca2eca5349 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:37:55.556918 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc0e17-566a-432e-a236-d4ca2eca5349 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:37:55.556918 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc0e17-566a-432e-a236-d4ca2eca5349 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:37:55.558369 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 606/1220] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:37:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:37:55.564485 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037030b-4ce1-44ae-8760-d7a0086c8504 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:37:55.564908 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037030b-4ce1-44ae-8760-d7a0086c8504 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:37:55.565041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037030b-4ce1-44ae-8760-d7a0086c8504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:37:55.585548 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-a037030b-4ce1-44ae-8760-d7a0086c8504 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:37:55.586943 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-a037030b-4ce1-44ae-8760-d7a0086c8504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:37:55.664175 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 615/1221] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:37:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 22:37:55.687358 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc0e17-566a-432e-a236-d4ca2eca5349 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:37:55.799510 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc0e17-566a-432e-a236-d4ca2eca5349 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:37:55.838120 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704b1b22-3963-4556-b7b0-2d3956ca3ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:37:55.838806 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b1b22-3963-4556-b7b0-2d3956ca3ca6 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:37:55.839162 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b1b22-3963-4556-b7b0-2d3956ca3ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:37:55.839447 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b1b22-3963-4556-b7b0-2d3956ca3ca6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:37:55.850674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704b1b22-3963-4556-b7b0-2d3956ca3ca6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:37:55.855393 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-704b1b22-3963-4556-b7b0-2d3956ca3ca6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:37:55.855393 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:37:55.855393 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:37:55.855393 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:37:55.855393 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:37:55.855393 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:37:55.855393 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:37:55.855393 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:37:55.855393 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:37:55.855393 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:37:55.855393 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:37:55.855393 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:37:55.855393 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:37:55.855393 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:37:55.855393 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:37:55.855393 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:37:55.856511 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:37:55.856511 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:37:55.856511 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:37:55.856511 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:37:55.856511 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:37:55.856511 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:37:55.856511 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:37:55.856511 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:37:55.856511 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:37:55.856511 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:37:55.856511 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:37:55.856511 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:37:55.856511 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:37:55.856511 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:37:55.856511 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:37:55.856511 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:37:55.857499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:37:55.857499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:37:55.857499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:37:55.857499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:37:55.857499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:37:55.857499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:37:55.857499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:37:55.857499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:37:55.857499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:37:55.857499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:37:55.857499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:37:55.857499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:37:55.857499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 607/1222] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 10 22:37:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:37:55.863605 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a037030b-4ce1-44ae-8760-d7a0086c8504 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:37:55.968660 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a037030b-4ce1-44ae-8760-d7a0086c8504 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:37:55.994050 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:37:56.008234 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f465af8-5a2f-4167-af94-73391e1910f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:37:56.008933 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f465af8-5a2f-4167-af94-73391e1910f9 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:37:56.009266 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f465af8-5a2f-4167-af94-73391e1910f9 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:37:56.009573 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f465af8-5a2f-4167-af94-73391e1910f9 None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:37:56.010377 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f465af8-5a2f-4167-af94-73391e1910f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:37:56.010662 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f465af8-5a2f-4167-af94-73391e1910f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:37:56.012215 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f465af8-5a2f-4167-af94-73391e1910f9 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:37:56.023265 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 616/1223] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 10 22:37:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:37:56.030787 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704b1b22-3963-4556-b7b0-2d3956ca3ca6 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:37:56.081190 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:37:56.142201 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704b1b22-3963-4556-b7b0-2d3956ca3ca6 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:37:56.177292 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fbe692-df93-427c-a132-b7b1b5d084c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:37:56.177967 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fbe692-df93-427c-a132-b7b1b5d084c1 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:37:56.178286 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fbe692-df93-427c-a132-b7b1b5d084c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:37:56.178593 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fbe692-df93-427c-a132-b7b1b5d084c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:37:56.190277 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fbe692-df93-427c-a132-b7b1b5d084c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:37:56.191733 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fbe692-df93-427c-a132-b7b1b5d084c1 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:37:56.211491 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-90fbe692-df93-427c-a132-b7b1b5d084c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:37:56.211491 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:37:56.211491 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:37:56.211491 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:37:56.211491 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:37:56.211491 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:37:56.211491 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:37:56.211491 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:37:56.211491 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:37:56.211491 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:37:56.211491 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:37:56.211491 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:37:56.211491 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:37:56.211491 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:37:56.211491 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:37:56.211491 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:37:56.212735 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:37:56.212735 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:37:56.212735 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:37:56.212735 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:37:56.212735 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:37:56.212735 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:37:56.212735 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:37:56.212735 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:37:56.212735 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:37:56.212735 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:37:56.212735 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:37:56.212735 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:37:56.212735 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:37:56.212735 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:37:56.212735 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:37:56.212735 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:37:56.212735 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:37:56.213672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:37:56.213672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:37:56.213672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:37:56.213672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 608/1224] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:37:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 22:37:56.220461 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f465af8-5a2f-4167-af94-73391e1910f9 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:37:56.336047 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f465af8-5a2f-4167-af94-73391e1910f9 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:37:56.372577 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dab4c7f-f529-4199-9512-13d5ad4b7fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:37:56.373194 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab4c7f-f529-4199-9512-13d5ad4b7fb6 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:37:56.374140 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab4c7f-f529-4199-9512-13d5ad4b7fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:37:56.374637 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab4c7f-f529-4199-9512-13d5ad4b7fb6 None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:37:56.375408 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dab4c7f-f529-4199-9512-13d5ad4b7fb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:37:56.375723 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dab4c7f-f529-4199-9512-13d5ad4b7fb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:37:56.377099 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dab4c7f-f529-4199-9512-13d5ad4b7fb6 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:37:56.389043 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 617/1225] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:37:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:37:56.395258 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fbe692-df93-427c-a132-b7b1b5d084c1 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:37:56.494363 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:37:56.502167 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fbe692-df93-427c-a132-b7b1b5d084c1 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:37:56.535756 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bafab70-ecf2-49dd-bf8e-71b0ad0ab07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:37:56.536388 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafab70-ecf2-49dd-bf8e-71b0ad0ab07a None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:37:56.536660 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafab70-ecf2-49dd-bf8e-71b0ad0ab07a None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:37:56.536926 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafab70-ecf2-49dd-bf8e-71b0ad0ab07a None admin] PATH_INFO: `/v3/users/placement` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:37:56.544729 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bafab70-ecf2-49dd-bf8e-71b0ad0ab07a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:37:56.547001 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bafab70-ecf2-49dd-bf8e-71b0ad0ab07a None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:37:56.557754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-0bafab70-ecf2-49dd-bf8e-71b0ad0ab07a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 22:37:56.557754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:37:56.557754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:37:56.557754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:37:56.557754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:37:56.557754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:37:56.557754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:37:56.557754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:37:56.557754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:37:56.557754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:37:56.557754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:37:56.557754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:37:56.557754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:37:56.557754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:37:56.557754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:37:56.557754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:37:56.559402 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:37:56.559402 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:37:56.559402 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:37:56.559402 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:37:56.559402 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:37:56.559402 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:37:56.559402 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:37:56.559402 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:37:56.559402 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:37:56.559402 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:37:56.559402 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:37:56.559402 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:37:56.559402 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:37:56.559402 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:37:56.559402 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:37:56.559402 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:37:56.559402 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:37:56.559402 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:37:56.560443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:37:56.560443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:37:56.560443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:37:56.560443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:37:56.560443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:37:56.560443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:37:56.560443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:37:56.560443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:37:56.560443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 22:37:56.560443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:37:56.560443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 609/1226] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:37:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:37:56.566690 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dab4c7f-f529-4199-9512-13d5ad4b7fb6 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:37:56.617510 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:37:56.674204 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dab4c7f-f529-4199-9512-13d5ad4b7fb6 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:37:56.711979 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce491d5-89ef-404a-a8cc-181ac0c12fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:37:56.712653 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce491d5-89ef-404a-a8cc-181ac0c12fac None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:37:56.712930 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce491d5-89ef-404a-a8cc-181ac0c12fac None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:37:56.713200 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce491d5-89ef-404a-a8cc-181ac0c12fac None admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:37:56.713955 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce491d5-89ef-404a-a8cc-181ac0c12fac None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:37:56.714255 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce491d5-89ef-404a-a8cc-181ac0c12fac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:37:56.715678 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce491d5-89ef-404a-a8cc-181ac0c12fac None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:37:56.734784 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 618/1227] 10.222.0.76 () {56 vars in 1143 bytes} [Wed Aug 10 22:37:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:37:56.740696 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bafab70-ecf2-49dd-bf8e-71b0ad0ab07a None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:37:56.851959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bafab70-ecf2-49dd-bf8e-71b0ad0ab07a None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:37:56.888676 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b02f5b-6a78-4578-945f-0d5c1eed2401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:37:56.889378 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b02f5b-6a78-4578-945f-0d5c1eed2401 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:37:56.889654 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b02f5b-6a78-4578-945f-0d5c1eed2401 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:37:56.889924 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b02f5b-6a78-4578-945f-0d5c1eed2401 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:37:56.902898 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b02f5b-6a78-4578-945f-0d5c1eed2401 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:37:56.905075 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b02f5b-6a78-4578-945f-0d5c1eed2401 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:37:56.922024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-a8b02f5b-6a78-4578-945f-0d5c1eed2401 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:37:56.922024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:37:56.922024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:37:56.922024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:37:56.922024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:37:56.922024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:37:56.922024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:37:56.922024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:37:56.922024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:37:56.922024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:37:56.922024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:37:56.922024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:37:56.922024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:37:56.922024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:37:56.922024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:37:56.922024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:37:56.923304 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:37:56.923304 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:37:56.923304 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:37:56.923304 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:37:56.923304 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:37:56.923304 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:37:56.923304 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:37:56.923304 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:37:56.923304 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:37:56.923304 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:37:56.923304 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:37:56.923304 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:37:56.923304 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:37:56.923304 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:37:56.923304 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:37:56.923304 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:37:56.923304 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:37:56.924389 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:37:56.924389 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:37:56.924389 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:37:56.924389 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 610/1228] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:37:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:37:56.929144 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce491d5-89ef-404a-a8cc-181ac0c12fac None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:37:57.065227 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce491d5-89ef-404a-a8cc-181ac0c12fac None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:37:57.067750 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:37:57.101273 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42caf2-a9cf-4519-9deb-05edf234082c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:37:57.101897 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42caf2-a9cf-4519-9deb-05edf234082c None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:37:57.102226 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42caf2-a9cf-4519-9deb-05edf234082c None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:37:57.102450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42caf2-a9cf-4519-9deb-05edf234082c None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:37:57.103087 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42caf2-a9cf-4519-9deb-05edf234082c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:37:57.103391 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42caf2-a9cf-4519-9deb-05edf234082c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:37:57.104768 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42caf2-a9cf-4519-9deb-05edf234082c None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:37:57.117886 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 619/1229] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:37:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:37:57.124647 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b02f5b-6a78-4578-945f-0d5c1eed2401 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:37:57.135517 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:37:57.239983 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b02f5b-6a78-4578-945f-0d5c1eed2401 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:37:57.280948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75028723-e20b-4ccf-8a20-27081089aba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:37:57.281637 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75028723-e20b-4ccf-8a20-27081089aba8 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:37:57.281899 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75028723-e20b-4ccf-8a20-27081089aba8 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:37:57.282166 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75028723-e20b-4ccf-8a20-27081089aba8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:37:57.289074 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75028723-e20b-4ccf-8a20-27081089aba8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:37:57.291013 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75028723-e20b-4ccf-8a20-27081089aba8 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:37:57.302290 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-75028723-e20b-4ccf-8a20-27081089aba8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:37:57.302290 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:37:57.302290 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:37:57.302290 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:37:57.302290 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:37:57.302290 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:37:57.302290 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:37:57.302290 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:37:57.302290 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:37:57.302290 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:37:57.302290 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:37:57.302290 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:37:57.302290 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:37:57.302290 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:37:57.302290 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:37:57.303524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:37:57.303524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:37:57.303524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:37:57.303524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:37:57.303524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:37:57.303524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:37:57.303524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:37:57.303524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:37:57.303524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:37:57.303524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:37:57.303524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:37:57.303524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:37:57.303524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:37:57.303524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:37:57.303524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:37:57.303524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:37:57.303524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:37:57.304662 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:37:57.304662 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:37:57.304662 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:37:57.304662 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:37:57.304662 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:37:57.304662 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:37:57.304662 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:37:57.304662 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:37:57.304662 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 611/1230] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:37:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:37:57.309343 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42caf2-a9cf-4519-9deb-05edf234082c None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:37:57.418479 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42caf2-a9cf-4519-9deb-05edf234082c None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:37:57.455641 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71086e62-c8f0-4ae5-a677-4e242edc85e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:37:57.456252 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71086e62-c8f0-4ae5-a677-4e242edc85e5 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:37:57.456535 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71086e62-c8f0-4ae5-a677-4e242edc85e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:37:57.456812 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71086e62-c8f0-4ae5-a677-4e242edc85e5 None admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:37:57.457522 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71086e62-c8f0-4ae5-a677-4e242edc85e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:37:57.457833 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71086e62-c8f0-4ae5-a677-4e242edc85e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:37:57.459293 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71086e62-c8f0-4ae5-a677-4e242edc85e5 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:37:57.471980 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 620/1231] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:37:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:37:57.478126 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75028723-e20b-4ccf-8a20-27081089aba8 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:37:57.550331 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:37:57.590867 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75028723-e20b-4ccf-8a20-27081089aba8 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:37:57.630843 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a4ca2b-3317-461a-8a3d-a7ce9fcdc9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:37:57.631508 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a4ca2b-3317-461a-8a3d-a7ce9fcdc9b2 None admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:37:57.631782 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a4ca2b-3317-461a-8a3d-a7ce9fcdc9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:37:57.632051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a4ca2b-3317-461a-8a3d-a7ce9fcdc9b2 None admin] PATH_INFO: `/v3/projects/3caa2ea846f7449b8c483788fc5846c8/users/9f46dee0f73e49eab54c11556e3b86bf/roles/5dfd0f952cd641bbb2257860c4c892ac` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:37:57.659162 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a4ca2b-3317-461a-8a3d-a7ce9fcdc9b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=3caa2ea846f7449b8c483788fc5846c8, user_id=9f46dee0f73e49eab54c11556e3b86bf, role_id=5dfd0f952cd641bbb2257860c4c892ac)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:37:57.660802 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a4ca2b-3317-461a-8a3d-a7ce9fcdc9b2 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:37:57.675968 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 612/1232] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 10 22:37:57 2022] PUT /identity/v3/projects/3caa2ea846f7449b8c483788fc5846c8/users/9f46dee0f73e49eab54c11556e3b86bf/roles/5dfd0f952cd641bbb2257860c4c892ac => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:37:59.757694 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079b365-fab7-429f-8cc3-ad5778c2381e None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:37:59.758305 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079b365-fab7-429f-8cc3-ad5778c2381e None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:37:59.758305 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079b365-fab7-429f-8cc3-ad5778c2381e None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:37:59.759709 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 621/1233] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:37:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:37:59.764999 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c55d3-3755-4492-ad5a-6752743479e3 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:37:59.765494 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c55d3-3755-4492-ad5a-6752743479e3 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:37:59.765601 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c55d3-3755-4492-ad5a-6752743479e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:37:59.784094 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-fe7c55d3-3755-4492-ad5a-6752743479e3 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:37:59.785472 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-fe7c55d3-3755-4492-ad5a-6752743479e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:37:59.857622 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 613/1234] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:37:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:37:59.881153 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e079b365-fab7-429f-8cc3-ad5778c2381e None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:37:59.909677 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:37:59.991550 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e079b365-fab7-429f-8cc3-ad5778c2381e None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:00.047910 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a43c5-2899-4bd0-b2da-10779d7f71e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:00.048671 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a43c5-2899-4bd0-b2da-10779d7f71e2 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:00.049055 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a43c5-2899-4bd0-b2da-10779d7f71e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:00.049477 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a43c5-2899-4bd0-b2da-10779d7f71e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:00.061118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a43c5-2899-4bd0-b2da-10779d7f71e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:38:00.066521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-d47a43c5-2899-4bd0-b2da-10779d7f71e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:38:00.066521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:38:00.066521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:38:00.066521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:38:00.066521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:38:00.066521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:38:00.066521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:38:00.066521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:38:00.066521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:38:00.066521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:38:00.066521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:38:00.066521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:38:00.066521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:38:00.066521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:38:00.066521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:38:00.066521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:38:00.067858 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:38:00.067858 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:38:00.067858 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:38:00.067858 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:38:00.067858 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:38:00.067858 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:38:00.067858 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:38:00.067858 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:38:00.067858 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:38:00.067858 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:38:00.067858 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:38:00.067858 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:38:00.067858 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:38:00.067858 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:38:00.067858 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:38:00.067858 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:38:00.069658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:38:00.069658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:38:00.069658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:38:00.069658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:38:00.069658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:38:00.069658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:38:00.069658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:38:00.069658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:38:00.069658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:38:00.069658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:38:00.069658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 22:38:00.069658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:38:00.069658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 622/1235] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 10 22:37:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:38:00.074819 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c55d3-3755-4492-ad5a-6752743479e3 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:00.123656 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:00.181114 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c55d3-3755-4492-ad5a-6752743479e3 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:00.224180 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dd878e-dbc8-4490-a5e4-0ab89d16b0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:00.224797 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd878e-dbc8-4490-a5e4-0ab89d16b0d7 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:00.225074 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd878e-dbc8-4490-a5e4-0ab89d16b0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:00.225373 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd878e-dbc8-4490-a5e4-0ab89d16b0d7 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:00.226036 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dd878e-dbc8-4490-a5e4-0ab89d16b0d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:38:00.226345 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dd878e-dbc8-4490-a5e4-0ab89d16b0d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:00.227703 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dd878e-dbc8-4490-a5e4-0ab89d16b0d7 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:00.245112 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 614/1236] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 10 22:38:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:38:00.246597 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a43c5-2899-4bd0-b2da-10779d7f71e2 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:00.342021 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:00.365528 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a43c5-2899-4bd0-b2da-10779d7f71e2 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:00.404899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03063fb6-6446-475a-a260-c6b2c92dae2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:00.405633 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03063fb6-6446-475a-a260-c6b2c92dae2a None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:00.405966 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03063fb6-6446-475a-a260-c6b2c92dae2a None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:00.406306 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03063fb6-6446-475a-a260-c6b2c92dae2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:00.420497 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03063fb6-6446-475a-a260-c6b2c92dae2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:00.422074 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03063fb6-6446-475a-a260-c6b2c92dae2a None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:00.442038 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-03063fb6-6446-475a-a260-c6b2c92dae2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:38:00.442038 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:38:00.442038 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:38:00.442038 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:38:00.442038 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:38:00.442038 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:38:00.442038 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:38:00.442038 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:38:00.442038 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:38:00.442038 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:38:00.442038 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:38:00.442038 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:38:00.442038 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:38:00.442038 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:38:00.442038 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:38:00.442038 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:38:00.443032 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:38:00.443032 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:38:00.443032 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:38:00.443032 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:38:00.443032 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:38:00.443032 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:38:00.443032 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:38:00.443032 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:38:00.443032 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:38:00.443032 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:38:00.443032 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:38:00.443032 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:38:00.443032 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:38:00.443032 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:38:00.443032 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:38:00.443032 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:38:00.443032 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:38:00.444288 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:38:00.444288 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:38:00.444288 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:38:00.444288 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 623/1237] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:38:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:38:00.448952 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dd878e-dbc8-4490-a5e4-0ab89d16b0d7 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:00.565176 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dd878e-dbc8-4490-a5e4-0ab89d16b0d7 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:00.576319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:00.609026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8c088e-d57f-499f-bc55-0c5f75b2eecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:00.609608 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c088e-d57f-499f-bc55-0c5f75b2eecc None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:00.609891 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c088e-d57f-499f-bc55-0c5f75b2eecc None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:00.610168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c088e-d57f-499f-bc55-0c5f75b2eecc None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:00.610813 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8c088e-d57f-499f-bc55-0c5f75b2eecc None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:38:00.611120 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8c088e-d57f-499f-bc55-0c5f75b2eecc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:00.612500 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8c088e-d57f-499f-bc55-0c5f75b2eecc None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:00.628221 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 615/1238] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:38:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:38:00.633958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03063fb6-6446-475a-a260-c6b2c92dae2a None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:00.739419 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03063fb6-6446-475a-a260-c6b2c92dae2a None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:00.772831 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97497da3-cf61-4bcb-8cc9-a8311fd1a75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:00.773511 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97497da3-cf61-4bcb-8cc9-a8311fd1a75b None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:00.773750 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97497da3-cf61-4bcb-8cc9-a8311fd1a75b None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:00.774020 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97497da3-cf61-4bcb-8cc9-a8311fd1a75b None admin] PATH_INFO: `/v3/users/placement` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:00.777415 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:00.783492 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97497da3-cf61-4bcb-8cc9-a8311fd1a75b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:00.784959 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97497da3-cf61-4bcb-8cc9-a8311fd1a75b None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:00.797039 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-97497da3-cf61-4bcb-8cc9-a8311fd1a75b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 22:38:00.797039 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:38:00.797039 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:38:00.797039 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:38:00.797039 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:38:00.797039 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:38:00.797039 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:38:00.797039 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:38:00.797039 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:38:00.797039 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:38:00.797039 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:38:00.797039 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:38:00.797039 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:38:00.797039 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:38:00.797039 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:38:00.797039 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:38:00.798363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:38:00.798363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:38:00.798363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:38:00.798363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:38:00.798363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:38:00.798363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:38:00.798363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:38:00.798363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:38:00.798363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:38:00.798363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:38:00.798363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:38:00.798363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:38:00.798363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:38:00.798363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:38:00.798363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:38:00.798363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:38:00.798363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:38:00.798363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:38:00.799334 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:38:00.799334 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:38:00.799334 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:38:00.799334 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:38:00.799334 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:38:00.799334 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:38:00.799334 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:38:00.799334 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:38:00.799334 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 22:38:00.799334 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:38:00.799334 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 624/1239] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:38:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:38:00.804364 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8c088e-d57f-499f-bc55-0c5f75b2eecc None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:00.916091 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8c088e-d57f-499f-bc55-0c5f75b2eecc None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:00.954208 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c8c0e-5daa-4a0d-8afb-9d1faebc5e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:00.954846 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c8c0e-5daa-4a0d-8afb-9d1faebc5e9b None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:00.955131 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c8c0e-5daa-4a0d-8afb-9d1faebc5e9b None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:00.955398 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c8c0e-5daa-4a0d-8afb-9d1faebc5e9b None admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:00.956149 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c8c0e-5daa-4a0d-8afb-9d1faebc5e9b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:38:00.956425 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c8c0e-5daa-4a0d-8afb-9d1faebc5e9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:00.957916 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c8c0e-5daa-4a0d-8afb-9d1faebc5e9b None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:00.977685 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 616/1240] 10.222.0.76 () {56 vars in 1143 bytes} [Wed Aug 10 22:38:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:38:00.983711 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97497da3-cf61-4bcb-8cc9-a8311fd1a75b None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:01.113797 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97497da3-cf61-4bcb-8cc9-a8311fd1a75b None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:01.151980 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9730ca-998a-42c5-9c6c-d3ab5033bbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:01.152758 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9730ca-998a-42c5-9c6c-d3ab5033bbb1 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:01.153176 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9730ca-998a-42c5-9c6c-d3ab5033bbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:01.154108 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9730ca-998a-42c5-9c6c-d3ab5033bbb1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:01.225162 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:01.232621 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9730ca-998a-42c5-9c6c-d3ab5033bbb1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:01.234110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9730ca-998a-42c5-9c6c-d3ab5033bbb1 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:01.341858 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 625/1241] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:38:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 22:38:01.347638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c8c0e-5daa-4a0d-8afb-9d1faebc5e9b None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:01.377864 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:01.458973 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c8c0e-5daa-4a0d-8afb-9d1faebc5e9b None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:01.495774 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b88214-272c-460c-8c3c-50a118b157b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:01.496609 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b88214-272c-460c-8c3c-50a118b157b5 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:01.497682 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b88214-272c-460c-8c3c-50a118b157b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:01.497988 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b88214-272c-460c-8c3c-50a118b157b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:01.510716 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b88214-272c-460c-8c3c-50a118b157b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:01.512171 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b88214-272c-460c-8c3c-50a118b157b5 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:01.528691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-f0b88214-272c-460c-8c3c-50a118b157b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:38:01.528691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:38:01.528691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:38:01.528691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:38:01.528691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:38:01.528691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:38:01.528691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:38:01.528691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:38:01.528691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:38:01.528691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:38:01.528691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:38:01.528691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:38:01.528691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:38:01.528691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:38:01.528691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:38:01.528691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:38:01.530041 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:38:01.530041 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:38:01.530041 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:38:01.530041 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:38:01.530041 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:38:01.530041 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:38:01.530041 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:38:01.530041 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:38:01.530041 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:38:01.530041 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:38:01.530041 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:38:01.530041 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:38:01.530041 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:38:01.530041 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:38:01.530041 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:38:01.530041 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:38:01.530041 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:38:01.530964 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:38:01.530964 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:38:01.530964 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:38:01.530964 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 617/1242] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:38:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:38:01.536032 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9730ca-998a-42c5-9c6c-d3ab5033bbb1 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:01.643043 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:01.649962 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9730ca-998a-42c5-9c6c-d3ab5033bbb1 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:01.690458 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f99384-22ee-444e-b3b5-8ca89f675d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:01.691448 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f99384-22ee-444e-b3b5-8ca89f675d45 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:01.691775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f99384-22ee-444e-b3b5-8ca89f675d45 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:01.692054 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f99384-22ee-444e-b3b5-8ca89f675d45 None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:01.692865 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f99384-22ee-444e-b3b5-8ca89f675d45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:38:01.693208 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f99384-22ee-444e-b3b5-8ca89f675d45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:01.695079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f99384-22ee-444e-b3b5-8ca89f675d45 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:01.709165 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 626/1243] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:38:01.714927 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b88214-272c-460c-8c3c-50a118b157b5 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:01.802742 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:01.826025 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b88214-272c-460c-8c3c-50a118b157b5 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:01.864587 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b36471-d233-4f2c-9593-348b47b9ab97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:01.865329 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b36471-d233-4f2c-9593-348b47b9ab97 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:01.865683 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b36471-d233-4f2c-9593-348b47b9ab97 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:01.865960 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b36471-d233-4f2c-9593-348b47b9ab97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:01.873443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b36471-d233-4f2c-9593-348b47b9ab97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:01.874901 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b36471-d233-4f2c-9593-348b47b9ab97 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:01.886209 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-25b36471-d233-4f2c-9593-348b47b9ab97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:38:01.886209 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:38:01.886209 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:38:01.886209 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:38:01.886209 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:38:01.886209 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:38:01.886209 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:38:01.886209 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:38:01.886209 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:38:01.886209 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:38:01.886209 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:38:01.886209 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:38:01.886209 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:38:01.886209 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:38:01.886209 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:38:01.887863 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:38:01.887863 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:38:01.887863 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:38:01.887863 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:38:01.887863 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:38:01.887863 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:38:01.887863 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:38:01.887863 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:38:01.887863 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:38:01.887863 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:38:01.887863 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:38:01.887863 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:38:01.887863 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:38:01.887863 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:38:01.887863 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:38:01.887863 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:38:01.887863 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:38:01.889689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:38:01.889689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:38:01.889689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:38:01.889689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:38:01.889689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:38:01.889689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:38:01.889689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:38:01.889689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:38:01.892912 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 618/1244] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:38:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:38:01.894383 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f99384-22ee-444e-b3b5-8ca89f675d45 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:02.013368 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f99384-22ee-444e-b3b5-8ca89f675d45 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:02.057732 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed946c2-bf6b-4491-976c-9fe004ae95ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:02.058302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed946c2-bf6b-4491-976c-9fe004ae95ba None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:02.058586 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed946c2-bf6b-4491-976c-9fe004ae95ba None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:02.058894 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed946c2-bf6b-4491-976c-9fe004ae95ba None admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:02.060435 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed946c2-bf6b-4491-976c-9fe004ae95ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:38:02.060744 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed946c2-bf6b-4491-976c-9fe004ae95ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:02.062486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed946c2-bf6b-4491-976c-9fe004ae95ba None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:02.075460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 627/1245] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:38:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:38:02.082053 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b36471-d233-4f2c-9593-348b47b9ab97 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:02.190643 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b36471-d233-4f2c-9593-348b47b9ab97 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:02.229187 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162d7a6-eea6-4046-90f3-7bef9e2777fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:02.229807 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162d7a6-eea6-4046-90f3-7bef9e2777fb None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:02.230112 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162d7a6-eea6-4046-90f3-7bef9e2777fb None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:02.230362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162d7a6-eea6-4046-90f3-7bef9e2777fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:02.231754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6162d7a6-eea6-4046-90f3-7bef9e2777fb None admin] RBAC: Adding query filter params (role.id=5dfd0f952cd641bbb2257860c4c892ac, scope.project.id=3caa2ea846f7449b8c483788fc5846c8, user.id=9f46dee0f73e49eab54c11556e3b86bf) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:38:02.232141 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6162d7a6-eea6-4046-90f3-7bef9e2777fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:02.233839 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6162d7a6-eea6-4046-90f3-7bef9e2777fb None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:02.240591 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 619/1246] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:38:02 2022] GET /identity/v3/role_assignments?user.id=9f46dee0f73e49eab54c11556e3b86bf&scope.project.id=3caa2ea846f7449b8c483788fc5846c8&role.id=5dfd0f952cd641bbb2257860c4c892ac => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:38:04.450801 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0223a17-4952-458d-846f-8de92d5e400b None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:04.451356 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0223a17-4952-458d-846f-8de92d5e400b None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:04.451679 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0223a17-4952-458d-846f-8de92d5e400b None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:04.453382 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 628/1247] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:38:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:38:04.459032 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f908a-9c98-440f-847e-9cf6b60d71f1 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:04.459569 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f908a-9c98-440f-847e-9cf6b60d71f1 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:04.459569 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f908a-9c98-440f-847e-9cf6b60d71f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:04.467146 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:04.481434 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-4d6f908a-9c98-440f-847e-9cf6b60d71f1 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:38:04.482722 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-4d6f908a-9c98-440f-847e-9cf6b60d71f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:38:04.563723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 620/1248] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:38:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:38:04.588223 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0223a17-4952-458d-846f-8de92d5e400b None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:04.610690 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:04.719446 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0223a17-4952-458d-846f-8de92d5e400b None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:04.760050 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d19c25-784d-4810-8e06-93030078442e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:04.760704 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d19c25-784d-4810-8e06-93030078442e None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:04.760989 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d19c25-784d-4810-8e06-93030078442e None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:04.761266 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d19c25-784d-4810-8e06-93030078442e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:04.774637 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d19c25-784d-4810-8e06-93030078442e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:38:04.781300 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-00d19c25-784d-4810-8e06-93030078442e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:38:04.781300 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:38:04.781300 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:38:04.781300 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:38:04.781300 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:38:04.781300 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:38:04.781300 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:38:04.781300 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:38:04.781300 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:38:04.781300 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:38:04.781300 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:38:04.781300 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:38:04.781300 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:38:04.781300 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:38:04.781300 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:38:04.781300 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:38:04.782528 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:38:04.782528 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:38:04.782528 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:38:04.782528 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:38:04.782528 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:38:04.782528 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:38:04.782528 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:38:04.782528 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:38:04.782528 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:38:04.782528 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:38:04.782528 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:38:04.782528 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:38:04.782528 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:38:04.782528 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:38:04.782528 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:38:04.782528 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:38:04.783493 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:38:04.783493 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:38:04.783493 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:38:04.783493 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:38:04.783493 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:38:04.783493 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:38:04.783493 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:38:04.783493 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:38:04.783493 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:38:04.783493 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:38:04.783493 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:38:04.783493 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:38:04.783493 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 629/1249] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:38:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:38:04.789872 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f908a-9c98-440f-847e-9cf6b60d71f1 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:04.907974 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f908a-9c98-440f-847e-9cf6b60d71f1 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:04.940073 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:04.947602 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5329b-21b3-4499-8e3f-b9ac42676b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:04.948160 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5329b-21b3-4499-8e3f-b9ac42676b34 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:04.948451 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5329b-21b3-4499-8e3f-b9ac42676b34 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:04.949183 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5329b-21b3-4499-8e3f-b9ac42676b34 None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:04.950629 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f5329b-21b3-4499-8e3f-b9ac42676b34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:38:04.951076 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f5329b-21b3-4499-8e3f-b9ac42676b34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:04.952505 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f5329b-21b3-4499-8e3f-b9ac42676b34 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:04.961458 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 621/1250] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:38:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:38:04.969043 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d19c25-784d-4810-8e06-93030078442e None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:05.061956 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:05.086967 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d19c25-784d-4810-8e06-93030078442e None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:05.124445 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df7542-da62-4185-9aa5-aaef88d7b4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:05.125070 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df7542-da62-4185-9aa5-aaef88d7b4cb None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:05.125402 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df7542-da62-4185-9aa5-aaef88d7b4cb None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:05.125675 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df7542-da62-4185-9aa5-aaef88d7b4cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:05.135522 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df7542-da62-4185-9aa5-aaef88d7b4cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:05.136955 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df7542-da62-4185-9aa5-aaef88d7b4cb None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:05.152354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-09df7542-da62-4185-9aa5-aaef88d7b4cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:38:05.152354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:38:05.152354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:38:05.152354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:38:05.152354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:38:05.152354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:38:05.152354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:38:05.152354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:38:05.152354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:38:05.152354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:38:05.152354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:38:05.152354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:38:05.152354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:38:05.152354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:38:05.152354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:38:05.152354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:38:05.159318 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:38:05.159318 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:38:05.159318 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:38:05.159318 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:38:05.159318 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:38:05.159318 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:38:05.159318 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:38:05.159318 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:38:05.159318 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:38:05.159318 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:38:05.159318 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:38:05.159318 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:38:05.159318 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:38:05.159318 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:38:05.159318 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:38:05.159318 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:38:05.159318 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:38:05.160313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:38:05.160313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:38:05.160313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:38:05.160313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 630/1251] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:38:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:38:05.160560 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5329b-21b3-4499-8e3f-b9ac42676b34 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:05.269668 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5329b-21b3-4499-8e3f-b9ac42676b34 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:05.309988 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a495b3-472f-48cd-a894-4dae97912660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:05.310614 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a495b3-472f-48cd-a894-4dae97912660 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:05.310974 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a495b3-472f-48cd-a894-4dae97912660 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:05.311250 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a495b3-472f-48cd-a894-4dae97912660 None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:05.311940 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a495b3-472f-48cd-a894-4dae97912660 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:38:05.312306 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a495b3-472f-48cd-a894-4dae97912660 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:05.313747 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a495b3-472f-48cd-a894-4dae97912660 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:05.326738 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 622/1252] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:38:05.333542 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df7542-da62-4185-9aa5-aaef88d7b4cb None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:05.451226 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df7542-da62-4185-9aa5-aaef88d7b4cb None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:05.482594 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:05.491283 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0e06a-7952-4b0c-b80b-b86d369feed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:05.491930 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0e06a-7952-4b0c-b80b-b86d369feed2 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:05.492209 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0e06a-7952-4b0c-b80b-b86d369feed2 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:05.492475 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0e06a-7952-4b0c-b80b-b86d369feed2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:05.502102 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0e06a-7952-4b0c-b80b-b86d369feed2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:05.503767 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0e06a-7952-4b0c-b80b-b86d369feed2 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:05.516567 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-e9a0e06a-7952-4b0c-b80b-b86d369feed2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 22:38:05.516567 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:38:05.516567 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:38:05.516567 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:38:05.516567 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:38:05.516567 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:38:05.516567 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:38:05.516567 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:38:05.516567 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:38:05.516567 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:38:05.516567 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:38:05.516567 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:38:05.516567 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:38:05.516567 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:38:05.516567 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:38:05.516567 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:38:05.517811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:38:05.517811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:38:05.517811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:38:05.517811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:38:05.517811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:38:05.517811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:38:05.517811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:38:05.517811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:38:05.517811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:38:05.517811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:38:05.517811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:38:05.517811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:38:05.517811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:38:05.517811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:38:05.517811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:38:05.517811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:38:05.517811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:38:05.517811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:38:05.518997 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:38:05.518997 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:38:05.518997 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:38:05.518997 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:38:05.518997 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:38:05.518997 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:38:05.518997 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:38:05.518997 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:38:05.518997 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 22:38:05.518997 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:38:05.518997 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 631/1253] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:38:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:38:05.523697 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a495b3-472f-48cd-a894-4dae97912660 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:05.589629 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:05.641218 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a495b3-472f-48cd-a894-4dae97912660 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:05.680139 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d20f52-e902-4652-8b71-66a7d7c25fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:05.680745 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d20f52-e902-4652-8b71-66a7d7c25fb1 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:05.681029 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d20f52-e902-4652-8b71-66a7d7c25fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:05.681343 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d20f52-e902-4652-8b71-66a7d7c25fb1 None admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:05.682079 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d20f52-e902-4652-8b71-66a7d7c25fb1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:38:05.682382 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d20f52-e902-4652-8b71-66a7d7c25fb1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:05.683892 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d20f52-e902-4652-8b71-66a7d7c25fb1 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:05.702902 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 623/1254] 10.222.0.76 () {56 vars in 1143 bytes} [Wed Aug 10 22:38:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:38:05.708790 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0e06a-7952-4b0c-b80b-b86d369feed2 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:05.831719 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0e06a-7952-4b0c-b80b-b86d369feed2 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:05.871068 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d695c3-1078-4a7e-a3ee-4a291e4cf6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:05.872444 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d695c3-1078-4a7e-a3ee-4a291e4cf6d8 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:05.872716 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d695c3-1078-4a7e-a3ee-4a291e4cf6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:05.872993 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d695c3-1078-4a7e-a3ee-4a291e4cf6d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:05.935474 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:05.949237 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d695c3-1078-4a7e-a3ee-4a291e4cf6d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:05.951538 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d695c3-1078-4a7e-a3ee-4a291e4cf6d8 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:06.074817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 632/1255] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:38:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 22:38:06.081829 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d20f52-e902-4652-8b71-66a7d7c25fb1 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:06.199298 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d20f52-e902-4652-8b71-66a7d7c25fb1 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:06.222636 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:06.252516 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed38a751-6d14-42bb-b444-a33d3a8adee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:06.253509 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38a751-6d14-42bb-b444-a33d3a8adee7 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:06.254058 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38a751-6d14-42bb-b444-a33d3a8adee7 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:06.254534 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38a751-6d14-42bb-b444-a33d3a8adee7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:06.267141 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed38a751-6d14-42bb-b444-a33d3a8adee7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:06.268616 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed38a751-6d14-42bb-b444-a33d3a8adee7 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:06.285113 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-ed38a751-6d14-42bb-b444-a33d3a8adee7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:38:06.285113 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:38:06.285113 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:38:06.285113 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:38:06.285113 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:38:06.285113 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:38:06.285113 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:38:06.285113 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:38:06.285113 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:38:06.285113 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:38:06.285113 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:38:06.285113 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:38:06.285113 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:38:06.285113 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:38:06.285113 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:38:06.285113 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:38:06.286272 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:38:06.286272 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:38:06.286272 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:38:06.286272 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:38:06.286272 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:38:06.286272 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:38:06.286272 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:38:06.286272 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:38:06.286272 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:38:06.286272 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:38:06.286272 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:38:06.286272 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:38:06.286272 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:38:06.286272 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:38:06.286272 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:38:06.286272 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:38:06.286272 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:38:06.287194 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:38:06.287194 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:38:06.287194 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:38:06.291289 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 624/1256] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:38:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:38:06.292374 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d695c3-1078-4a7e-a3ee-4a291e4cf6d8 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:06.394575 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:06.408376 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d695c3-1078-4a7e-a3ee-4a291e4cf6d8 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:06.446253 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b43415-c601-43f2-be56-a8a475ddf284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:06.446849 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b43415-c601-43f2-be56-a8a475ddf284 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:06.447130 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b43415-c601-43f2-be56-a8a475ddf284 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:06.447399 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b43415-c601-43f2-be56-a8a475ddf284 None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:06.448044 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b43415-c601-43f2-be56-a8a475ddf284 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:38:06.448333 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b43415-c601-43f2-be56-a8a475ddf284 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:06.449818 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b43415-c601-43f2-be56-a8a475ddf284 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:06.461795 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 633/1257] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:38:06.468579 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed38a751-6d14-42bb-b444-a33d3a8adee7 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:06.581598 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed38a751-6d14-42bb-b444-a33d3a8adee7 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:06.620424 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f7195b-1a47-4fb3-a226-0c0ab773670b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:06.621159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7195b-1a47-4fb3-a226-0c0ab773670b None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:06.621474 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7195b-1a47-4fb3-a226-0c0ab773670b None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:06.621750 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7195b-1a47-4fb3-a226-0c0ab773670b None admin] PATH_INFO: `/v3/projects/service` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:06.628939 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f7195b-1a47-4fb3-a226-0c0ab773670b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:06.630481 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f7195b-1a47-4fb3-a226-0c0ab773670b None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:06.641793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-22f7195b-1a47-4fb3-a226-0c0ab773670b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:38:06.641793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:38:06.641793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:38:06.641793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:38:06.641793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:38:06.641793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:38:06.641793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:38:06.641793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:38:06.641793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:38:06.641793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:38:06.641793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:38:06.641793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:38:06.641793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:38:06.641793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:38:06.641793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:38:06.642761 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:38:06.642761 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:38:06.642761 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:38:06.642761 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:38:06.642761 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:38:06.642761 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:38:06.642761 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:38:06.642761 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:38:06.642761 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:38:06.642761 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:38:06.642761 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:38:06.642761 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:38:06.642761 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:38:06.642761 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:38:06.642761 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:38:06.642761 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:38:06.642761 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:38:06.643694 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:38:06.643694 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:38:06.643694 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:38:06.643694 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:38:06.643694 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:38:06.643694 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:38:06.643694 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:38:06.643694 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:38:06.643694 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 625/1258] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:38:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:38:06.649518 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b43415-c601-43f2-be56-a8a475ddf284 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:06.756125 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b43415-c601-43f2-be56-a8a475ddf284 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:06.793724 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d4db0-1093-4851-a852-3272e8812511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:06.794996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d4db0-1093-4851-a852-3272e8812511 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:06.795273 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d4db0-1093-4851-a852-3272e8812511 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:06.795575 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d4db0-1093-4851-a852-3272e8812511 None admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:06.796392 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d4db0-1093-4851-a852-3272e8812511 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:38:06.796698 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d4db0-1093-4851-a852-3272e8812511 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:06.798236 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d4db0-1093-4851-a852-3272e8812511 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:06.811222 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 634/1259] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:38:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:38:06.817242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f7195b-1a47-4fb3-a226-0c0ab773670b None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:06.846185 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:06.928843 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f7195b-1a47-4fb3-a226-0c0ab773670b None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:06.965373 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4620986-72cb-4e34-8670-d8ce05537f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:06.965995 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4620986-72cb-4e34-8670-d8ce05537f8a None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:06.967242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4620986-72cb-4e34-8670-d8ce05537f8a None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:06.967576 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4620986-72cb-4e34-8670-d8ce05537f8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:06.968287 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4620986-72cb-4e34-8670-d8ce05537f8a None admin] RBAC: Adding query filter params (role.id=6641e95fdc1f4c098ebc372bb692d227, scope.project.id=3caa2ea846f7449b8c483788fc5846c8, user.id=9f46dee0f73e49eab54c11556e3b86bf) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:38:06.968600 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4620986-72cb-4e34-8670-d8ce05537f8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:06.970071 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4620986-72cb-4e34-8670-d8ce05537f8a None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:06.975747 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 626/1260] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:38:06 2022] GET /identity/v3/role_assignments?user.id=9f46dee0f73e49eab54c11556e3b86bf&scope.project.id=3caa2ea846f7449b8c483788fc5846c8&role.id=6641e95fdc1f4c098ebc372bb692d227 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:38:09.010719 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22273e-d3b1-4cf3-87ac-75d702311994 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:09.011276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22273e-d3b1-4cf3-87ac-75d702311994 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:09.011370 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22273e-d3b1-4cf3-87ac-75d702311994 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:09.012889 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 635/1261] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:38:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:38:09.018949 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8453c-0810-42e7-b3bc-e2768bdb43bc None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:09.019382 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8453c-0810-42e7-b3bc-e2768bdb43bc None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:09.019497 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8453c-0810-42e7-b3bc-e2768bdb43bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:09.038329 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-3ee8453c-0810-42e7-b3bc-e2768bdb43bc None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:38:09.039506 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-3ee8453c-0810-42e7-b3bc-e2768bdb43bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:38:09.106280 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 627/1262] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:38:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:38:09.129582 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff22273e-d3b1-4cf3-87ac-75d702311994 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:09.140449 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:09.231007 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff22273e-d3b1-4cf3-87ac-75d702311994 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:09.263913 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641e588-45bc-44d6-b12b-399a7f914be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:09.265319 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641e588-45bc-44d6-b12b-399a7f914be2 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:09.265613 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641e588-45bc-44d6-b12b-399a7f914be2 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:09.265888 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641e588-45bc-44d6-b12b-399a7f914be2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:09.275681 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a641e588-45bc-44d6-b12b-399a7f914be2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:38:09.281343 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-a641e588-45bc-44d6-b12b-399a7f914be2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:38:09.281343 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:38:09.281343 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:38:09.281343 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:38:09.281343 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:38:09.281343 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:38:09.281343 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:38:09.281343 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:38:09.281343 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:38:09.281343 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:38:09.281343 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:38:09.281343 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:38:09.281343 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:38:09.281343 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:38:09.281343 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:38:09.281343 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:38:09.282476 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:38:09.282476 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:38:09.282476 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:38:09.282476 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:38:09.282476 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:38:09.282476 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:38:09.282476 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:38:09.282476 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:38:09.282476 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:38:09.282476 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:38:09.282476 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:38:09.282476 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:38:09.282476 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:38:09.282476 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:38:09.282476 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:38:09.282476 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:38:09.283518 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:38:09.283518 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:38:09.283518 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:38:09.283518 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:38:09.283518 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:38:09.283518 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:38:09.283518 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:38:09.283518 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:38:09.283518 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:38:09.283518 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:38:09.283518 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:38:09.283518 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:38:09.283518 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 636/1263] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:38:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:38:09.288443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8453c-0810-42e7-b3bc-e2768bdb43bc None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:09.306952 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:09.388355 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8453c-0810-42e7-b3bc-e2768bdb43bc None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:09.422732 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93898159-9b29-4085-a1aa-d59ceb24af1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:09.423334 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93898159-9b29-4085-a1aa-d59ceb24af1d None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:09.423609 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93898159-9b29-4085-a1aa-d59ceb24af1d None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:09.423877 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93898159-9b29-4085-a1aa-d59ceb24af1d None admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:09.424526 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93898159-9b29-4085-a1aa-d59ceb24af1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:38:09.424822 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93898159-9b29-4085-a1aa-d59ceb24af1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:09.426218 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93898159-9b29-4085-a1aa-d59ceb24af1d None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:09.434545 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 628/1264] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:38:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:38:09.440397 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641e588-45bc-44d6-b12b-399a7f914be2 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:09.506234 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:09.539710 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641e588-45bc-44d6-b12b-399a7f914be2 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:09.572718 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1173032a-0277-4dac-bfdf-3470ebe79dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:09.573471 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173032a-0277-4dac-bfdf-3470ebe79dd2 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:09.573785 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173032a-0277-4dac-bfdf-3470ebe79dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:09.573930 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173032a-0277-4dac-bfdf-3470ebe79dd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:09.584288 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1173032a-0277-4dac-bfdf-3470ebe79dd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:09.585788 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1173032a-0277-4dac-bfdf-3470ebe79dd2 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:09.601355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-1173032a-0277-4dac-bfdf-3470ebe79dd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:38:09.601355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:38:09.601355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:38:09.601355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:38:09.601355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:38:09.601355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:38:09.601355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:38:09.601355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:38:09.601355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:38:09.601355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:38:09.601355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:38:09.601355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:38:09.601355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:38:09.601355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:38:09.601355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:38:09.601355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:38:09.602524 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:38:09.602524 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:38:09.602524 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:38:09.602524 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:38:09.602524 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:38:09.602524 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:38:09.602524 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:38:09.602524 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:38:09.602524 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:38:09.602524 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:38:09.602524 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:38:09.602524 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:38:09.602524 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:38:09.602524 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:38:09.602524 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:38:09.602524 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:38:09.602524 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:38:09.603636 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:38:09.603636 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:38:09.603636 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:38:09.603636 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 637/1265] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:38:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:38:09.608187 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93898159-9b29-4085-a1aa-d59ceb24af1d None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:09.687656 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:09.707500 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93898159-9b29-4085-a1aa-d59ceb24af1d None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:09.739298 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdace20-9824-4921-91ba-352d02d6a319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:09.739893 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdace20-9824-4921-91ba-352d02d6a319 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:09.740166 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdace20-9824-4921-91ba-352d02d6a319 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:09.740432 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdace20-9824-4921-91ba-352d02d6a319 None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:09.741056 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdace20-9824-4921-91ba-352d02d6a319 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:38:09.741457 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdace20-9824-4921-91ba-352d02d6a319 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:09.742810 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdace20-9824-4921-91ba-352d02d6a319 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:09.755311 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 629/1266] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:38:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:38:09.760928 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1173032a-0277-4dac-bfdf-3470ebe79dd2 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:09.858313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1173032a-0277-4dac-bfdf-3470ebe79dd2 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:09.878709 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:09.893503 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c4de9-02c1-426f-a60e-9ca11aecb487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:09.894119 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c4de9-02c1-426f-a60e-9ca11aecb487 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:09.894397 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c4de9-02c1-426f-a60e-9ca11aecb487 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:09.894667 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c4de9-02c1-426f-a60e-9ca11aecb487 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:09.901447 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c4de9-02c1-426f-a60e-9ca11aecb487 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:09.902880 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c4de9-02c1-426f-a60e-9ca11aecb487 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:09.914123 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-6f2c4de9-02c1-426f-a60e-9ca11aecb487 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 22:38:09.914123 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:38:09.914123 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:38:09.914123 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:38:09.914123 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:38:09.914123 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:38:09.914123 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:38:09.914123 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:38:09.914123 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:38:09.914123 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:38:09.914123 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:38:09.914123 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:38:09.914123 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:38:09.914123 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:38:09.914123 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:38:09.914123 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:38:09.915232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:38:09.915232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:38:09.915232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:38:09.915232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:38:09.915232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:38:09.915232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:38:09.915232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:38:09.915232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:38:09.915232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:38:09.915232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:38:09.915232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:38:09.915232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:38:09.915232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:38:09.915232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:38:09.915232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:38:09.915232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:38:09.915232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:38:09.915232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:38:09.916637 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:38:09.916637 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:38:09.916637 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:38:09.916637 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:38:09.916637 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:38:09.916637 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:38:09.916637 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:38:09.916637 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:38:09.916637 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 22:38:09.916637 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:38:09.916637 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 638/1267] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:38:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:38:09.921843 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdace20-9824-4921-91ba-352d02d6a319 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:10.024323 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdace20-9824-4921-91ba-352d02d6a319 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:10.060200 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f93c5e-867c-486c-9989-8142c9c91ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:10.060803 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f93c5e-867c-486c-9989-8142c9c91ddf None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:10.061077 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f93c5e-867c-486c-9989-8142c9c91ddf None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:10.062130 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f93c5e-867c-486c-9989-8142c9c91ddf None admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:10.062841 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f93c5e-867c-486c-9989-8142c9c91ddf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:38:10.063141 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f93c5e-867c-486c-9989-8142c9c91ddf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:10.064563 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f93c5e-867c-486c-9989-8142c9c91ddf None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:10.073170 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:10.081589 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 630/1268] 10.222.0.76 () {56 vars in 1143 bytes} [Wed Aug 10 22:38:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:38:10.087302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c4de9-02c1-426f-a60e-9ca11aecb487 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:10.189600 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c4de9-02c1-426f-a60e-9ca11aecb487 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:10.220985 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ff5ce-e2f3-4d45-bfc2-2a68e45e7b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:10.221632 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ff5ce-e2f3-4d45-bfc2-2a68e45e7b0f None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:10.221941 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ff5ce-e2f3-4d45-bfc2-2a68e45e7b0f None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:10.222212 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ff5ce-e2f3-4d45-bfc2-2a68e45e7b0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:10.232195 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ff5ce-e2f3-4d45-bfc2-2a68e45e7b0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:10.234235 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ff5ce-e2f3-4d45-bfc2-2a68e45e7b0f None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:10.247668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-113ff5ce-e2f3-4d45-bfc2-2a68e45e7b0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:38:10.247668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:38:10.247668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:38:10.247668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:38:10.247668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:38:10.247668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:38:10.247668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:38:10.247668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:38:10.247668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:38:10.247668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:38:10.247668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:38:10.247668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:38:10.247668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:38:10.247668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:38:10.247668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:38:10.247668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:38:10.253369 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:38:10.253369 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:38:10.253369 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:38:10.253369 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:38:10.253369 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:38:10.253369 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:38:10.253369 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:38:10.253369 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:38:10.253369 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:38:10.253369 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:38:10.253369 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:38:10.253369 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:38:10.253369 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:38:10.253369 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:38:10.253369 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:38:10.253369 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:38:10.253369 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:38:10.254513 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:38:10.254513 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:38:10.254513 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:38:10.254513 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 639/1269] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:38:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:38:10.255399 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f93c5e-867c-486c-9989-8142c9c91ddf None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:10.355327 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f93c5e-867c-486c-9989-8142c9c91ddf None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:10.388433 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8951a-9aee-478d-b44a-0ccb1a06cc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:10.389034 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8951a-9aee-478d-b44a-0ccb1a06cc91 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:10.389334 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8951a-9aee-478d-b44a-0ccb1a06cc91 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:10.389647 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8951a-9aee-478d-b44a-0ccb1a06cc91 None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:10.390483 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda8951a-9aee-478d-b44a-0ccb1a06cc91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:38:10.390780 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda8951a-9aee-478d-b44a-0ccb1a06cc91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:10.392148 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda8951a-9aee-478d-b44a-0ccb1a06cc91 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:10.405998 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 631/1270] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:38:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:38:10.411965 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ff5ce-e2f3-4d45-bfc2-2a68e45e7b0f None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:10.446006 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:10.515782 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ff5ce-e2f3-4d45-bfc2-2a68e45e7b0f None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:10.548882 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93535600-3b2d-411a-b9d7-b33429e73476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:10.549536 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93535600-3b2d-411a-b9d7-b33429e73476 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:10.549818 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93535600-3b2d-411a-b9d7-b33429e73476 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:10.550091 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93535600-3b2d-411a-b9d7-b33429e73476 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:10.556492 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93535600-3b2d-411a-b9d7-b33429e73476 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:10.558098 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93535600-3b2d-411a-b9d7-b33429e73476 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:10.569908 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-93535600-3b2d-411a-b9d7-b33429e73476 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:38:10.569908 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:38:10.569908 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:38:10.569908 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:38:10.569908 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:38:10.569908 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:38:10.569908 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:38:10.569908 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:38:10.569908 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:38:10.569908 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:38:10.569908 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:38:10.569908 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:38:10.569908 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:38:10.569908 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:38:10.569908 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:38:10.570929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:38:10.570929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:38:10.570929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:38:10.570929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:38:10.570929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:38:10.570929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:38:10.570929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:38:10.570929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:38:10.570929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:38:10.570929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:38:10.570929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:38:10.570929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:38:10.570929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:38:10.570929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:38:10.570929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:38:10.570929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:38:10.570929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:38:10.571870 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:38:10.571870 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:38:10.571870 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:38:10.571870 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:38:10.571870 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:38:10.571870 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:38:10.571870 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:38:10.571870 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:38:10.571870 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 640/1271] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:38:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:38:10.576836 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8951a-9aee-478d-b44a-0ccb1a06cc91 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:10.634348 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:10.681251 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8951a-9aee-478d-b44a-0ccb1a06cc91 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:10.714232 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d7aef9-74cf-4789-8c7d-79df6dd8fa78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:10.714770 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7aef9-74cf-4789-8c7d-79df6dd8fa78 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:10.715047 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7aef9-74cf-4789-8c7d-79df6dd8fa78 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:10.715317 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7aef9-74cf-4789-8c7d-79df6dd8fa78 None admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:10.716001 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d7aef9-74cf-4789-8c7d-79df6dd8fa78 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:38:10.716305 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d7aef9-74cf-4789-8c7d-79df6dd8fa78 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:10.717780 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d7aef9-74cf-4789-8c7d-79df6dd8fa78 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:10.729484 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 632/1272] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:38:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:38:10.736115 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93535600-3b2d-411a-b9d7-b33429e73476 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:10.830766 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:10.839582 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93535600-3b2d-411a-b9d7-b33429e73476 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:10.872595 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98370b27-5708-445b-bc67-5ed499f2122a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:10.873244 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98370b27-5708-445b-bc67-5ed499f2122a None admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:10.873558 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98370b27-5708-445b-bc67-5ed499f2122a None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:10.873831 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98370b27-5708-445b-bc67-5ed499f2122a None admin] PATH_INFO: `/v3/projects/3caa2ea846f7449b8c483788fc5846c8/users/9f46dee0f73e49eab54c11556e3b86bf/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:10.876344 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98370b27-5708-445b-bc67-5ed499f2122a None admin] RBAC: Authorizing `identity:create_grant(project_id=3caa2ea846f7449b8c483788fc5846c8, user_id=9f46dee0f73e49eab54c11556e3b86bf, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:10.877808 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98370b27-5708-445b-bc67-5ed499f2122a None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:10.889797 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 641/1273] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 10 22:38:10 2022] PUT /identity/v3/projects/3caa2ea846f7449b8c483788fc5846c8/users/9f46dee0f73e49eab54c11556e3b86bf/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:38:12.865611 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b3859-ad7b-49af-89bd-faf7cff73c94 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:12.866141 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b3859-ad7b-49af-89bd-faf7cff73c94 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:12.866141 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b3859-ad7b-49af-89bd-faf7cff73c94 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:12.867507 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 633/1274] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:38:12.872502 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718de64-7974-451c-ac76-2617ca656649 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:12.872934 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718de64-7974-451c-ac76-2617ca656649 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:12.873058 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718de64-7974-451c-ac76-2617ca656649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:12.892603 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-f718de64-7974-451c-ac76-2617ca656649 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:38:12.893831 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-f718de64-7974-451c-ac76-2617ca656649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:38:12.959683 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 642/1275] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:38:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:38:12.982415 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b3859-ad7b-49af-89bd-faf7cff73c94 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:13.098145 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b3859-ad7b-49af-89bd-faf7cff73c94 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:13.115474 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:13.132198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f32dcce-a401-432a-96e5-140e8a3acf00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:13.132752 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32dcce-a401-432a-96e5-140e8a3acf00 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:13.133027 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32dcce-a401-432a-96e5-140e8a3acf00 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:13.133322 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32dcce-a401-432a-96e5-140e8a3acf00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:13.143843 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f32dcce-a401-432a-96e5-140e8a3acf00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 22:38:13.148377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-4f32dcce-a401-432a-96e5-140e8a3acf00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:38:13.148377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:38:13.148377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:38:13.148377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:38:13.148377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:38:13.148377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:38:13.148377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:38:13.148377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:38:13.148377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:38:13.148377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:38:13.148377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:38:13.148377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:38:13.148377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 22:38:13.148377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 22:38:13.148377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 22:38:13.148377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 22:38:13.153340 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 22:38:13.153340 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 22:38:13.153340 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 22:38:13.153340 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 22:38:13.153340 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:38:13.153340 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:38:13.153340 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:38:13.153340 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:38:13.153340 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:38:13.153340 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:38:13.153340 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:38:13.153340 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:38:13.153340 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:38:13.153340 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:38:13.153340 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:38:13.153340 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:38:13.154415 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:38:13.154415 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:38:13.154415 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:38:13.154415 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:38:13.154415 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 22:38:13.154415 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 22:38:13.154415 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 22:38:13.154415 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 22:38:13.154415 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 22:38:13.154415 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 22:38:13.154415 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 22:38:13.154415 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:38:13.154415 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 634/1276] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 10 22:38:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:38:13.155693 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718de64-7974-451c-ac76-2617ca656649 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:13.235598 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:13.255377 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718de64-7974-451c-ac76-2617ca656649 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:13.289118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a569947-40e0-4d0d-a808-4ab3ad57b35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:13.289748 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a569947-40e0-4d0d-a808-4ab3ad57b35a None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:13.290031 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a569947-40e0-4d0d-a808-4ab3ad57b35a None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:13.290304 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a569947-40e0-4d0d-a808-4ab3ad57b35a None admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:13.290957 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a569947-40e0-4d0d-a808-4ab3ad57b35a None admin] RBAC: Adding query filter params (name=admin) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:38:13.291258 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a569947-40e0-4d0d-a808-4ab3ad57b35a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:13.292906 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a569947-40e0-4d0d-a808-4ab3ad57b35a None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:13.301512 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 643/1277] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 10 22:38:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:38:13.307216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f32dcce-a401-432a-96e5-140e8a3acf00 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:13.406213 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f32dcce-a401-432a-96e5-140e8a3acf00 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:13.443363 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10096a6-c0c1-4584-8fb6-4c7bf51bf833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:13.445851 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10096a6-c0c1-4584-8fb6-4c7bf51bf833 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:13.446131 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10096a6-c0c1-4584-8fb6-4c7bf51bf833 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:13.446408 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10096a6-c0c1-4584-8fb6-4c7bf51bf833 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:13.460159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10096a6-c0c1-4584-8fb6-4c7bf51bf833 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:13.461660 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10096a6-c0c1-4584-8fb6-4c7bf51bf833 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:13.477437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-b10096a6-c0c1-4584-8fb6-4c7bf51bf833 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:38:13.477437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:38:13.477437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:38:13.477437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:38:13.477437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:38:13.477437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:38:13.477437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:38:13.477437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:38:13.477437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:38:13.477437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:38:13.477437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:38:13.477437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:38:13.477437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:38:13.477437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:38:13.477437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:38:13.477437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:38:13.478592 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:38:13.478592 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:38:13.478592 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:38:13.478592 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:38:13.478592 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:38:13.478592 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:38:13.478592 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:38:13.478592 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:38:13.478592 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:38:13.478592 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:38:13.478592 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:38:13.478592 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:38:13.478592 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:38:13.478592 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:38:13.478592 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:38:13.478592 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:38:13.478592 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:38:13.479753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:38:13.479753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:38:13.479753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:38:13.479753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 635/1278] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:38:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:38:13.485145 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a569947-40e0-4d0d-a808-4ab3ad57b35a None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:13.584464 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a569947-40e0-4d0d-a808-4ab3ad57b35a None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:13.616590 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a71dd5-364c-4b45-87ee-495f6b79dc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:13.617193 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a71dd5-364c-4b45-87ee-495f6b79dc84 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:13.617499 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a71dd5-364c-4b45-87ee-495f6b79dc84 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:13.617772 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a71dd5-364c-4b45-87ee-495f6b79dc84 None admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:13.618399 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a71dd5-364c-4b45-87ee-495f6b79dc84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:38:13.618699 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a71dd5-364c-4b45-87ee-495f6b79dc84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:13.620061 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a71dd5-364c-4b45-87ee-495f6b79dc84 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:13.631956 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 644/1279] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:38:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:38:13.637796 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10096a6-c0c1-4584-8fb6-4c7bf51bf833 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:13.668391 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:13.735529 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10096a6-c0c1-4584-8fb6-4c7bf51bf833 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:13.769724 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392802b-43d0-41b9-bc92-b0eda90c724b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:13.770366 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392802b-43d0-41b9-bc92-b0eda90c724b None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:13.770646 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392802b-43d0-41b9-bc92-b0eda90c724b None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:13.770917 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392802b-43d0-41b9-bc92-b0eda90c724b None admin] PATH_INFO: `/v3/users/placement` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:13.780167 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b392802b-43d0-41b9-bc92-b0eda90c724b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:13.782887 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b392802b-43d0-41b9-bc92-b0eda90c724b None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:13.794362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-b392802b-43d0-41b9-bc92-b0eda90c724b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 22:38:13.794362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:38:13.794362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:38:13.794362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:38:13.794362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:38:13.794362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:38:13.794362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:38:13.794362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:38:13.794362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:38:13.794362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:38:13.794362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:38:13.794362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:38:13.794362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 22:38:13.794362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 22:38:13.794362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 22:38:13.794362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 22:38:13.795495 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:38:13.795495 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:38:13.795495 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 22:38:13.795495 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:38:13.795495 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 22:38:13.795495 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 22:38:13.795495 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:38:13.795495 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:38:13.795495 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:38:13.795495 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:38:13.795495 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:38:13.795495 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:38:13.795495 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:38:13.795495 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:38:13.795495 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:38:13.795495 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:38:13.795495 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:38:13.795495 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:38:13.796666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:38:13.796666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:38:13.796666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 22:38:13.796666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 22:38:13.796666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 22:38:13.796666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 22:38:13.796666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 22:38:13.796666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 22:38:13.796666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 22:38:13.796666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:38:13.796666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 636/1280] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:38:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:38:13.803704 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a71dd5-364c-4b45-87ee-495f6b79dc84 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:13.807646 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:13.903996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a71dd5-364c-4b45-87ee-495f6b79dc84 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:13.936580 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b553cc-083e-4f3b-8e9b-6613ad7a900f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:13.937120 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b553cc-083e-4f3b-8e9b-6613ad7a900f None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:13.937430 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b553cc-083e-4f3b-8e9b-6613ad7a900f None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:13.937701 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b553cc-083e-4f3b-8e9b-6613ad7a900f None admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:13.939032 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b553cc-083e-4f3b-8e9b-6613ad7a900f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:38:13.939327 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b553cc-083e-4f3b-8e9b-6613ad7a900f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:13.940744 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b553cc-083e-4f3b-8e9b-6613ad7a900f None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:13.956985 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 645/1281] 10.222.0.76 () {56 vars in 1143 bytes} [Wed Aug 10 22:38:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:38:13.962886 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392802b-43d0-41b9-bc92-b0eda90c724b None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:14.065086 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:14.074951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392802b-43d0-41b9-bc92-b0eda90c724b None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:14.109590 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26032ab-5fa5-4971-8c40-de6498f7b86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:14.110496 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26032ab-5fa5-4971-8c40-de6498f7b86b None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:14.110824 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26032ab-5fa5-4971-8c40-de6498f7b86b None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:14.111103 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26032ab-5fa5-4971-8c40-de6498f7b86b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:14.181793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26032ab-5fa5-4971-8c40-de6498f7b86b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:14.183268 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26032ab-5fa5-4971-8c40-de6498f7b86b None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:14.271008 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:14.281065 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 637/1282] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 10 22:38:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 22:38:14.287282 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b553cc-083e-4f3b-8e9b-6613ad7a900f None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:14.349740 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:14.387856 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b553cc-083e-4f3b-8e9b-6613ad7a900f None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:14.421468 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6989787d-c2c6-4ae3-80f7-8c0b3fe6dec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:14.422101 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6989787d-c2c6-4ae3-80f7-8c0b3fe6dec6 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:14.422889 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6989787d-c2c6-4ae3-80f7-8c0b3fe6dec6 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:14.423167 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6989787d-c2c6-4ae3-80f7-8c0b3fe6dec6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:14.433231 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6989787d-c2c6-4ae3-80f7-8c0b3fe6dec6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:14.434695 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6989787d-c2c6-4ae3-80f7-8c0b3fe6dec6 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:14.451363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-6989787d-c2c6-4ae3-80f7-8c0b3fe6dec6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:38:14.451363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:38:14.451363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:38:14.451363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:38:14.451363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:38:14.451363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:38:14.451363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:38:14.451363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:38:14.451363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:38:14.451363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:38:14.451363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:38:14.451363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:38:14.451363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 22:38:14.451363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 22:38:14.451363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 22:38:14.451363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 22:38:14.452559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:38:14.452559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:38:14.452559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:38:14.452559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:38:14.452559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:38:14.452559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:38:14.452559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:38:14.452559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:38:14.452559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:38:14.452559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:38:14.452559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:38:14.452559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:38:14.452559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:38:14.452559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:38:14.452559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:38:14.452559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:38:14.452559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 22:38:14.453711 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 22:38:14.453711 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 22:38:14.453711 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:38:14.453711 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 646/1283] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:38:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:38:14.458550 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26032ab-5fa5-4971-8c40-de6498f7b86b None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:14.555865 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26032ab-5fa5-4971-8c40-de6498f7b86b None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:14.589754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25458cb4-cf9f-4421-82ad-dfc2405aadd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:14.590351 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25458cb4-cf9f-4421-82ad-dfc2405aadd1 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:14.590628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25458cb4-cf9f-4421-82ad-dfc2405aadd1 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:14.590895 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25458cb4-cf9f-4421-82ad-dfc2405aadd1 None admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:14.591518 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25458cb4-cf9f-4421-82ad-dfc2405aadd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:38:14.591812 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25458cb4-cf9f-4421-82ad-dfc2405aadd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:14.593171 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25458cb4-cf9f-4421-82ad-dfc2405aadd1 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:14.604038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 638/1284] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 10 22:38:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:38:14.609706 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6989787d-c2c6-4ae3-80f7-8c0b3fe6dec6 None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:14.706812 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6989787d-c2c6-4ae3-80f7-8c0b3fe6dec6 None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:14.720378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:14.740794 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0953531-1e48-4028-981b-a5d13910e95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:14.741435 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0953531-1e48-4028-981b-a5d13910e95b None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:14.741707 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0953531-1e48-4028-981b-a5d13910e95b None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:14.741974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0953531-1e48-4028-981b-a5d13910e95b None admin] PATH_INFO: `/v3/projects/service` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:14.748318 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0953531-1e48-4028-981b-a5d13910e95b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:14.749783 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0953531-1e48-4028-981b-a5d13910e95b None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:14.760885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application [None req-c0953531-1e48-4028-981b-a5d13910e95b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:38:14.760885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:38:14.760885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:38:14.760885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:38:14.760885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:38:14.760885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:38:14.760885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:38:14.760885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:38:14.760885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:38:14.760885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:38:14.760885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:38:14.760885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:38:14.760885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 22:38:14.760885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 22:38:14.760885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 22:38:14.761974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 22:38:14.761974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:38:14.761974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:38:14.761974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:38:14.761974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:38:14.761974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:38:14.761974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:38:14.761974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:38:14.761974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application with Lock( Aug 10 22:38:14.761974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:38:14.761974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:38:14.761974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:38:14.761974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:38:14.761974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:38:14.761974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:38:14.761974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:38:14.761974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:38:14.763085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 22:38:14.763085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 22:38:14.763085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 22:38:14.763085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 22:38:14.763085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 22:38:14.763085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 22:38:14.763085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 22:38:14.763085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: ERROR keystone.server.flask.application Aug 10 22:38:14.763085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 647/1285] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 10 22:38:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:38:14.768045 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25458cb4-cf9f-4421-82ad-dfc2405aadd1 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:14.821154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:14.872343 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25458cb4-cf9f-4421-82ad-dfc2405aadd1 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:14.904425 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad5842-3fa9-40d8-8200-d842fd54bafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:14.905024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad5842-3fa9-40d8-8200-d842fd54bafe None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:14.905323 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad5842-3fa9-40d8-8200-d842fd54bafe None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:14.905599 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad5842-3fa9-40d8-8200-d842fd54bafe None admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:14.906280 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad5842-3fa9-40d8-8200-d842fd54bafe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:38:14.906585 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad5842-3fa9-40d8-8200-d842fd54bafe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:14.908832 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad5842-3fa9-40d8-8200-d842fd54bafe None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:14.920692 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 639/1286] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 10 22:38:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:38:14.927491 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0953531-1e48-4028-981b-a5d13910e95b None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:15.028941 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0953531-1e48-4028-981b-a5d13910e95b None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:15.063439 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e31f3-301c-48a2-9673-413c8ce929c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:15.064042 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e31f3-301c-48a2-9673-413c8ce929c0 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:15.064314 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e31f3-301c-48a2-9673-413c8ce929c0 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:15.064584 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e31f3-301c-48a2-9673-413c8ce929c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:15.065317 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e31f3-301c-48a2-9673-413c8ce929c0 None admin] RBAC: Adding query filter params (role.id=6641e95fdc1f4c098ebc372bb692d227, scope.project.id=3caa2ea846f7449b8c483788fc5846c8, user.id=9f46dee0f73e49eab54c11556e3b86bf) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:38:15.065627 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e31f3-301c-48a2-9673-413c8ce929c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:15.067049 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e31f3-301c-48a2-9673-413c8ce929c0 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:15.072681 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 648/1287] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 10 22:38:14 2022] GET /identity/v3/role_assignments?user.id=9f46dee0f73e49eab54c11556e3b86bf&scope.project.id=3caa2ea846f7449b8c483788fc5846c8&role.id=6641e95fdc1f4c098ebc372bb692d227 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:38:17.125985 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b770f-7342-4d79-b54d-a5c5b5b35e0a None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:17.126565 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b770f-7342-4d79-b54d-a5c5b5b35e0a None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:17.126565 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b770f-7342-4d79-b54d-a5c5b5b35e0a None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:17.128172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 640/1288] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:38:17.133360 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3e138-3682-4011-b1e9-a18aeb049071 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:17.133789 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3e138-3682-4011-b1e9-a18aeb049071 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:17.133922 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3e138-3682-4011-b1e9-a18aeb049071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:17.151901 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-62b3e138-3682-4011-b1e9-a18aeb049071 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:38:17.153091 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-62b3e138-3682-4011-b1e9-a18aeb049071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:38:17.182962 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:17.218934 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 649/1289] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:38:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:38:17.241817 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75b770f-7342-4d79-b54d-a5c5b5b35e0a None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:17.340404 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75b770f-7342-4d79-b54d-a5c5b5b35e0a None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:17.353501 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:17.377489 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07784ef8-8d65-4e80-8ace-3296a777c0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:17.378042 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07784ef8-8d65-4e80-8ace-3296a777c0d6 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:17.378319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07784ef8-8d65-4e80-8ace-3296a777c0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:17.378587 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07784ef8-8d65-4e80-8ace-3296a777c0d6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:17.379155 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07784ef8-8d65-4e80-8ace-3296a777c0d6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:17.380521 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07784ef8-8d65-4e80-8ace-3296a777c0d6 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:17.388987 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-07784ef8-8d65-4e80-8ace-3296a777c0d6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 22:38:17.388987 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:38:17.388987 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:38:17.388987 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:38:17.388987 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:38:17.388987 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:38:17.388987 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:38:17.388987 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:38:17.388987 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 22:38:17.388987 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:38:17.388987 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:38:17.388987 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:38:17.388987 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:38:17.388987 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:38:17.388987 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:38:17.388987 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:38:17.388987 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:38:17.390242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:38:17.390242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:38:17.390242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:38:17.390242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:38:17.390242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:38:17.390242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:38:17.390242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:38:17.390242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:38:17.390242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:38:17.390242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:38:17.390242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:38:17.390242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:38:17.390242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:38:17.390242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:38:17.390242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:38:17.390242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:38:17.391134 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:38:17.391134 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:38:17.391134 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:38:17.391134 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:38:17.391134 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:38:17.391134 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:38:17.391134 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:38:17.391134 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:38:17.391134 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:38:17.391134 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:38:17.391134 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:38:17.391134 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:38:17.391134 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 22:38:17.391134 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:38:17.391134 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 641/1290] 10.222.0.76 () {56 vars in 1117 bytes} [Wed Aug 10 22:38:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:38:17.396232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3e138-3682-4011-b1e9-a18aeb049071 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:17.494544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3e138-3682-4011-b1e9-a18aeb049071 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:17.527403 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28413b22-e96e-4ffd-a35c-bffe2b8ff2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:17.528001 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28413b22-e96e-4ffd-a35c-bffe2b8ff2ba None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:17.528278 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28413b22-e96e-4ffd-a35c-bffe2b8ff2ba None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:17.528544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28413b22-e96e-4ffd-a35c-bffe2b8ff2ba None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:17.529154 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28413b22-e96e-4ffd-a35c-bffe2b8ff2ba None admin] RBAC: Adding query filter params (name=placement) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:38:17.529481 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28413b22-e96e-4ffd-a35c-bffe2b8ff2ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:17.530836 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28413b22-e96e-4ffd-a35c-bffe2b8ff2ba None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:17.536086 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 650/1291] 10.222.0.76 () {56 vars in 1116 bytes} [Wed Aug 10 22:38:17 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:38:17.541861 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07784ef8-8d65-4e80-8ace-3296a777c0d6 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:17.639323 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07784ef8-8d65-4e80-8ace-3296a777c0d6 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:17.672748 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af130c-0525-43c8-a11b-3ccc19665ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:17.673372 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af130c-0525-43c8-a11b-3ccc19665ef0 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:17.673680 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af130c-0525-43c8-a11b-3ccc19665ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:17.673952 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af130c-0525-43c8-a11b-3ccc19665ef0 None admin] PATH_INFO: `/v3/services` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:17.674569 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af130c-0525-43c8-a11b-3ccc19665ef0 None admin] RBAC: Adding query filter params (type=placement) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:38:17.674864 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af130c-0525-43c8-a11b-3ccc19665ef0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:17.676220 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af130c-0525-43c8-a11b-3ccc19665ef0 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:17.680681 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 642/1292] 10.222.0.76 () {56 vars in 1116 bytes} [Wed Aug 10 22:38:17 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:38:19.657357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f7c480-6517-4a17-81c1-2f504b47a39c None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:19.657936 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f7c480-6517-4a17-81c1-2f504b47a39c None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:19.657936 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f7c480-6517-4a17-81c1-2f504b47a39c None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:19.659891 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 651/1293] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:38:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:38:19.664990 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478005b-67ac-47c2-ad20-a8d5a25e4538 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:19.665399 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478005b-67ac-47c2-ad20-a8d5a25e4538 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:19.665570 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478005b-67ac-47c2-ad20-a8d5a25e4538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:19.684447 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-8478005b-67ac-47c2-ad20-a8d5a25e4538 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:38:19.685602 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-8478005b-67ac-47c2-ad20-a8d5a25e4538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:38:19.714618 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:19.751624 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 643/1294] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:38:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:38:19.773844 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f7c480-6517-4a17-81c1-2f504b47a39c None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:19.804663 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:19.874878 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f7c480-6517-4a17-81c1-2f504b47a39c None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:19.907805 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56685298-0ffa-4f0d-8013-4bbed659beeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:19.908396 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56685298-0ffa-4f0d-8013-4bbed659beeb None admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:19.908854 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56685298-0ffa-4f0d-8013-4bbed659beeb None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:19.909134 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56685298-0ffa-4f0d-8013-4bbed659beeb None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:19.910024 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56685298-0ffa-4f0d-8013-4bbed659beeb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:19.911390 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56685298-0ffa-4f0d-8013-4bbed659beeb None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:19.925878 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 652/1295] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 10 22:38:19 2022] POST /identity/v3/services => generated 246 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:38:21.918029 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a9b76-512a-4b0b-b6d8-ea9f7298144e None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:21.918600 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a9b76-512a-4b0b-b6d8-ea9f7298144e None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:21.918600 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a9b76-512a-4b0b-b6d8-ea9f7298144e None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:21.919902 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 644/1296] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:38:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:38:21.925123 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ea2e7-326a-49b5-bcba-aa850867f7ae None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:21.925646 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ea2e7-326a-49b5-bcba-aa850867f7ae None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:21.925646 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ea2e7-326a-49b5-bcba-aa850867f7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:21.944228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-a37ea2e7-326a-49b5-bcba-aa850867f7ae None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:38:21.945470 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-a37ea2e7-326a-49b5-bcba-aa850867f7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:38:22.021303 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 653/1297] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:38:21 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:38:22.047162 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a9b76-512a-4b0b-b6d8-ea9f7298144e None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:22.152218 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a9b76-512a-4b0b-b6d8-ea9f7298144e None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:22.185717 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ee1bb-8b0b-42df-81b2-b5ab2100371f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:22.186265 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ee1bb-8b0b-42df-81b2-b5ab2100371f None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:22.186539 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ee1bb-8b0b-42df-81b2-b5ab2100371f None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:22.186805 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ee1bb-8b0b-42df-81b2-b5ab2100371f None admin] PATH_INFO: `/v3/services/placement` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:22.187380 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74ee1bb-8b0b-42df-81b2-b5ab2100371f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:22.188728 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74ee1bb-8b0b-42df-81b2-b5ab2100371f None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:22.197622 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-d74ee1bb-8b0b-42df-81b2-b5ab2100371f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 22:38:22.197622 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:38:22.197622 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:38:22.197622 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:38:22.197622 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:38:22.197622 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:38:22.197622 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:38:22.197622 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:38:22.197622 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 22:38:22.197622 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:38:22.197622 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:38:22.197622 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:38:22.197622 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:38:22.197622 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:38:22.197622 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:38:22.197622 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:38:22.197622 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:38:22.198864 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:38:22.198864 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:38:22.198864 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:38:22.198864 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:38:22.198864 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:38:22.198864 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:38:22.198864 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:38:22.198864 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:38:22.198864 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:38:22.198864 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:38:22.198864 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:38:22.198864 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:38:22.198864 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:38:22.198864 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:38:22.198864 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:38:22.198864 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:38:22.199897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:38:22.199897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:38:22.199897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:38:22.199897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:38:22.199897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:38:22.199897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:38:22.199897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:38:22.199897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:38:22.199897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:38:22.199897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:38:22.199897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:38:22.199897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:38:22.199897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 22:38:22.199897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:38:22.199897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 645/1298] 10.222.0.76 () {56 vars in 1117 bytes} [Wed Aug 10 22:38:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:38:22.204745 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ea2e7-326a-49b5-bcba-aa850867f7ae None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:22.224095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:22.302002 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ea2e7-326a-49b5-bcba-aa850867f7ae None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:22.335360 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21c578-4ff4-4251-954e-6c449979630e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:22.335965 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21c578-4ff4-4251-954e-6c449979630e None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:22.336247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21c578-4ff4-4251-954e-6c449979630e None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:22.336523 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21c578-4ff4-4251-954e-6c449979630e None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:22.337141 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c21c578-4ff4-4251-954e-6c449979630e None admin] RBAC: Adding query filter params (name=placement) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:38:22.337490 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c21c578-4ff4-4251-954e-6c449979630e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:22.338855 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c21c578-4ff4-4251-954e-6c449979630e None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:22.343814 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 654/1299] 10.222.0.76 () {56 vars in 1116 bytes} [Wed Aug 10 22:38:22 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:38:22.349992 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ee1bb-8b0b-42df-81b2-b5ab2100371f None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:22.383959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:22.454831 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ee1bb-8b0b-42df-81b2-b5ab2100371f None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:22.487714 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3405a6-29e1-4dd2-8102-649193ccc893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:22.488869 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3405a6-29e1-4dd2-8102-649193ccc893 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:22.489143 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3405a6-29e1-4dd2-8102-649193ccc893 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:22.489445 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3405a6-29e1-4dd2-8102-649193ccc893 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:22.490137 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3405a6-29e1-4dd2-8102-649193ccc893 None admin] RBAC: Adding query filter params (interface=public, service_id=ac70359b4f4f46b98e0b911ff165c052, region_id=RegionOne) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:38:22.490440 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3405a6-29e1-4dd2-8102-649193ccc893 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:22.491800 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3405a6-29e1-4dd2-8102-649193ccc893 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:22.496698 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 646/1300] 10.222.0.76 () {56 vars in 1251 bytes} [Wed Aug 10 22:38:22 2022] GET /identity/v3/endpoints?service_id=ac70359b4f4f46b98e0b911ff165c052&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:38:22.502361 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21c578-4ff4-4251-954e-6c449979630e None admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:22.581728 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:22.599741 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21c578-4ff4-4251-954e-6c449979630e None admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:22.632761 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf931d9-0f81-497d-8990-60247d001786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:22.633389 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf931d9-0f81-497d-8990-60247d001786 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:22.633678 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf931d9-0f81-497d-8990-60247d001786 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:22.633948 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf931d9-0f81-497d-8990-60247d001786 None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:22.634540 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf931d9-0f81-497d-8990-60247d001786 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:22.635899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf931d9-0f81-497d-8990-60247d001786 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:22.642434 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 655/1301] 10.222.0.76 () {56 vars in 1087 bytes} [Wed Aug 10 22:38:22 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:38:24.660858 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da50c1-7b8c-4bf3-8966-607dd1533328 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:24.661461 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da50c1-7b8c-4bf3-8966-607dd1533328 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:24.661461 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da50c1-7b8c-4bf3-8966-607dd1533328 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:24.662792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 647/1302] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:38:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:38:24.667935 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c06fc3-eb4f-42e0-a36f-c8b7d41a03e4 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:24.668362 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c06fc3-eb4f-42e0-a36f-c8b7d41a03e4 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:24.668482 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c06fc3-eb4f-42e0-a36f-c8b7d41a03e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:24.687257 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-54c06fc3-eb4f-42e0-a36f-c8b7d41a03e4 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:38:24.688383 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-54c06fc3-eb4f-42e0-a36f-c8b7d41a03e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:38:24.752865 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 656/1303] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:38:24 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 22:38:24.775543 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da50c1-7b8c-4bf3-8966-607dd1533328 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:24.871346 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:24.878702 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da50c1-7b8c-4bf3-8966-607dd1533328 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:24.912462 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018e268-c639-4ef0-aa34-9e535addc665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:24.913078 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018e268-c639-4ef0-aa34-9e535addc665 None admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:24.913395 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018e268-c639-4ef0-aa34-9e535addc665 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:24.913673 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018e268-c639-4ef0-aa34-9e535addc665 None admin] PATH_INFO: `/v3/services/placement` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:24.914250 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b018e268-c639-4ef0-aa34-9e535addc665 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:24.915619 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b018e268-c639-4ef0-aa34-9e535addc665 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:24.924308 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application [None req-b018e268-c639-4ef0-aa34-9e535addc665 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 22:38:24.924308 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:38:24.924308 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 22:38:24.924308 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 22:38:24.924308 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 22:38:24.924308 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 22:38:24.924308 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 22:38:24.924308 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:38:24.924308 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 22:38:24.924308 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:38:24.924308 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 22:38:24.924308 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:38:24.924308 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 22:38:24.924308 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 22:38:24.924308 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 22:38:24.924308 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 22:38:24.924308 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 22:38:24.925549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 22:38:24.925549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 22:38:24.925549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 22:38:24.925549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 22:38:24.925549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 22:38:24.925549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 22:38:24.925549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 22:38:24.925549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 22:38:24.925549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 22:38:24.925549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 22:38:24.925549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 22:38:24.925549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 22:38:24.925549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 22:38:24.925549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 22:38:24.925549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 22:38:24.925549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 22:38:24.927228 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 22:38:24.927228 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application with Lock( Aug 10 22:38:24.927228 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 22:38:24.927228 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self._enter() Aug 10 22:38:24.927228 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 22:38:24.927228 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 22:38:24.927228 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 22:38:24.927228 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application return self.creator() Aug 10 22:38:24.927228 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 22:38:24.927228 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application created_value = creator( Aug 10 22:38:24.927228 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 22:38:24.927228 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 22:38:24.927228 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 22:38:24.927228 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: ERROR keystone.server.flask.application Aug 10 22:38:24.927228 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 648/1304] 10.222.0.76 () {56 vars in 1117 bytes} [Wed Aug 10 22:38:24 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 22:38:24.932311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c06fc3-eb4f-42e0-a36f-c8b7d41a03e4 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:25.009773 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:25.032676 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c06fc3-eb4f-42e0-a36f-c8b7d41a03e4 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:25.066314 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10584fae-dd97-4a36-9442-11bc969718b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:25.067502 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10584fae-dd97-4a36-9442-11bc969718b9 None admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:25.067962 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10584fae-dd97-4a36-9442-11bc969718b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:25.068238 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10584fae-dd97-4a36-9442-11bc969718b9 None admin] PATH_INFO: `/v3/services` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:25.068858 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10584fae-dd97-4a36-9442-11bc969718b9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:38:25.069155 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10584fae-dd97-4a36-9442-11bc969718b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:25.070581 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10584fae-dd97-4a36-9442-11bc969718b9 None admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:25.075661 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 657/1305] 10.222.0.76 () {56 vars in 1116 bytes} [Wed Aug 10 22:38:24 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:38:25.081468 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018e268-c639-4ef0-aa34-9e535addc665 None admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:38:25.179819 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018e268-c639-4ef0-aa34-9e535addc665 None admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:38:25.213350 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51121973-416a-4f5f-ada4-2fc4451e31e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:38:25.213943 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51121973-416a-4f5f-ada4-2fc4451e31e6 None admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:38:25.214217 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51121973-416a-4f5f-ada4-2fc4451e31e6 None admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:38:25.214483 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51121973-416a-4f5f-ada4-2fc4451e31e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:38:25.215118 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51121973-416a-4f5f-ada4-2fc4451e31e6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:38:25.216467 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51121973-416a-4f5f-ada4-2fc4451e31e6 None admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:38:25.235572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 649/1306] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:38:25 2022] POST /identity/v3/endpoints => generated 331 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:38:35.073169 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=30 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:38:35.232184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:39:29.877085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e701844-08d0-4986-9b52-1313dd877567 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:39:29.878304 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e701844-08d0-4986-9b52-1313dd877567 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:39:29.879013 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e701844-08d0-4986-9b52-1313dd877567 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:39:29.885454 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 658/1307] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:39:29 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:39:29.887029 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f3155-784d-49ba-b14f-3cef4915f2d8 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:39:29.887534 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f3155-784d-49ba-b14f-3cef4915f2d8 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:39:29.887645 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f3155-784d-49ba-b14f-3cef4915f2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:39:30.012425 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-d04f3155-784d-49ba-b14f-3cef4915f2d8 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:39:30.013951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-d04f3155-784d-49ba-b14f-3cef4915f2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:39:30.140574 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 650/1308] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:39:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:39:30.191989 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8a4f1-4632-46e9-987f-9d24cac19f28 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:39:30.192197 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8a4f1-4632-46e9-987f-9d24cac19f28 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:39:30.192722 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8a4f1-4632-46e9-987f-9d24cac19f28 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:39:30.194203 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 659/1309] 10.222.0.76 () {54 vars in 965 bytes} [Wed Aug 10 22:39:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:39:30.203064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad651d1-726b-4abd-ad98-cb4a28aec608 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:39:30.203659 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad651d1-726b-4abd-ad98-cb4a28aec608 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:39:30.204073 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad651d1-726b-4abd-ad98-cb4a28aec608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:39:30.296849 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-1ad651d1-726b-4abd-ad98-cb4a28aec608 None None] MFA Rules not processed for user `7fcb5a674ff94b3d8f79efbbe4959069`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:39:30.298333 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-1ad651d1-726b-4abd-ad98-cb4a28aec608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:39:30.323055 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:39:30.388509 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 651/1310] 10.222.0.76 () {58 vars in 1062 bytes} [Wed Aug 10 22:39:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:39:30.396812 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8a4f1-4632-46e9-987f-9d24cac19f28 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:39:30.398646 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-48d8a4f1-4632-46e9-987f-9d24cac19f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:39:30.521514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8a4f1-4632-46e9-987f-9d24cac19f28 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:39:30.602134 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51529d-c50f-4009-8010-0a70b057070a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:39:30.602803 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51529d-c50f-4009-8010-0a70b057070a service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:39:30.603088 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51529d-c50f-4009-8010-0a70b057070a service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:39:30.603360 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51529d-c50f-4009-8010-0a70b057070a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:39:30.605616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-9b51529d-c50f-4009-8010-0a70b057070a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:39:30.664436 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:39:30.740529 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51529d-c50f-4009-8010-0a70b057070a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:39:30.742025 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51529d-c50f-4009-8010-0a70b057070a service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:39:30.764473 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 660/1311] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:39:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:39:33.136635 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60098bc-23b9-4f14-bf29-f1fa30e4830a None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:39:33.137186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60098bc-23b9-4f14-bf29-f1fa30e4830a None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:39:33.137303 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60098bc-23b9-4f14-bf29-f1fa30e4830a None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:39:33.138738 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 652/1312] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:39:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:39:33.143639 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d2aa8-792f-4297-bc28-de9a506fe3fc None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:39:33.144037 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d2aa8-792f-4297-bc28-de9a506fe3fc None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:39:33.144181 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d2aa8-792f-4297-bc28-de9a506fe3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:39:33.214779 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-767d2aa8-792f-4297-bc28-de9a506fe3fc None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:39:33.215977 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-767d2aa8-792f-4297-bc28-de9a506fe3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:39:33.310460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 661/1313] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:39:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:39:33.363261 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8778cc4-1a33-43ff-aa3c-ae9677a554ee None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:39:33.363546 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8778cc4-1a33-43ff-aa3c-ae9677a554ee None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:39:33.363820 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8778cc4-1a33-43ff-aa3c-ae9677a554ee None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:39:33.365164 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 653/1314] 10.222.0.76 () {54 vars in 965 bytes} [Wed Aug 10 22:39:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:39:33.371768 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602750c-0e5b-4cdc-9058-76e525ecaf75 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:39:33.372052 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602750c-0e5b-4cdc-9058-76e525ecaf75 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:39:33.372328 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602750c-0e5b-4cdc-9058-76e525ecaf75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:39:33.384412 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:39:33.450305 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-7602750c-0e5b-4cdc-9058-76e525ecaf75 None None] MFA Rules not processed for user `7fcb5a674ff94b3d8f79efbbe4959069`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:39:33.451514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-7602750c-0e5b-4cdc-9058-76e525ecaf75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:39:33.525668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 662/1315] 10.222.0.76 () {58 vars in 1062 bytes} [Wed Aug 10 22:39:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:39:33.534743 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8778cc4-1a33-43ff-aa3c-ae9677a554ee None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:39:33.536597 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-a8778cc4-1a33-43ff-aa3c-ae9677a554ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:39:33.639337 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8778cc4-1a33-43ff-aa3c-ae9677a554ee None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:39:33.710201 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:39:33.713426 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb777d-07f4-4d11-bb38-f31ad325cfa3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:39:33.714409 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb777d-07f4-4d11-bb38-f31ad325cfa3 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:39:33.714865 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb777d-07f4-4d11-bb38-f31ad325cfa3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:39:33.715295 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb777d-07f4-4d11-bb38-f31ad325cfa3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:39:33.716907 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-f2eb777d-07f4-4d11-bb38-f31ad325cfa3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:39:33.834686 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb777d-07f4-4d11-bb38-f31ad325cfa3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:39:33.836177 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb777d-07f4-4d11-bb38-f31ad325cfa3 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:39:33.855994 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 654/1316] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:39:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:39:36.420633 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f1e5d-35f9-4b51-aec8-8f87cd094a07 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:39:36.421171 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f1e5d-35f9-4b51-aec8-8f87cd094a07 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:39:36.421171 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f1e5d-35f9-4b51-aec8-8f87cd094a07 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:39:36.422546 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 663/1317] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:39:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:39:36.427607 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f653a-390a-463d-aa27-c60cf9f81264 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:39:36.428382 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f653a-390a-463d-aa27-c60cf9f81264 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:39:36.428484 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f653a-390a-463d-aa27-c60cf9f81264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:39:36.499141 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-993f653a-390a-463d-aa27-c60cf9f81264 None None] MFA Rules not processed for user `6e08ec69331f4a57817e3de09ab9221e`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:39:36.500461 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-993f653a-390a-463d-aa27-c60cf9f81264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:39:36.550494 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:39:36.595147 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 655/1318] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:39:36 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:39:36.635473 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07f1e5d-35f9-4b51-aec8-8f87cd094a07 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:39:36.637000 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-d07f1e5d-35f9-4b51-aec8-8f87cd094a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:39:36.740127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07f1e5d-35f9-4b51-aec8-8f87cd094a07 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:39:36.742972 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:39:36.840311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda3306-91cc-48b3-a69c-ffdb4447bbaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:39:36.840933 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda3306-91cc-48b3-a69c-ffdb4447bbaa service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:39:36.841210 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda3306-91cc-48b3-a69c-ffdb4447bbaa service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:39:36.841507 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda3306-91cc-48b3-a69c-ffdb4447bbaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:39:36.843146 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-bfda3306-91cc-48b3-a69c-ffdb4447bbaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:39:36.951410 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfda3306-91cc-48b3-a69c-ffdb4447bbaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:39:36.952840 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfda3306-91cc-48b3-a69c-ffdb4447bbaa service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:39:36.970876 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 664/1319] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:39:36 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:39:39.586618 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da43e7-7fd0-4314-bcc9-9c01775a09d1 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:39:39.587169 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da43e7-7fd0-4314-bcc9-9c01775a09d1 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:39:39.587169 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da43e7-7fd0-4314-bcc9-9c01775a09d1 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:39:39.589076 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 656/1320] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:39:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:39:39.594183 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a0321-614c-4ef6-aa6b-adc99d4cb071 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:39:39.594611 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a0321-614c-4ef6-aa6b-adc99d4cb071 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:39:39.594729 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a0321-614c-4ef6-aa6b-adc99d4cb071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:39:39.635014 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:39:39.660284 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-e46a0321-614c-4ef6-aa6b-adc99d4cb071 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:39:39.661541 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-e46a0321-614c-4ef6-aa6b-adc99d4cb071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:39:39.757328 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 665/1321] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:39:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:39:39.796372 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da43e7-7fd0-4314-bcc9-9c01775a09d1 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:39:39.797965 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-10da43e7-7fd0-4314-bcc9-9c01775a09d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:39:39.900813 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da43e7-7fd0-4314-bcc9-9c01775a09d1 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:39:39.973159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c3cf1-9d09-4256-98b7-fefe4716381a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:39:39.973788 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c3cf1-9d09-4256-98b7-fefe4716381a service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:39:39.974073 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c3cf1-9d09-4256-98b7-fefe4716381a service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:39:39.974350 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c3cf1-9d09-4256-98b7-fefe4716381a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:39:39.975947 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-f84c3cf1-9d09-4256-98b7-fefe4716381a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:39:39.983425 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:39:40.089821 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c3cf1-9d09-4256-98b7-fefe4716381a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:39:40.091359 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c3cf1-9d09-4256-98b7-fefe4716381a service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:39:40.110295 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 657/1322] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:39:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:39:43.060103 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a29893-b666-48ce-a962-86956c42c14b None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:39:43.060552 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a29893-b666-48ce-a962-86956c42c14b None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:39:43.060660 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a29893-b666-48ce-a962-86956c42c14b None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:39:43.062028 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 666/1323] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:39:43.067131 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb669d7-c8df-4b1c-b352-bd4f012dd9e7 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:39:43.067538 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb669d7-c8df-4b1c-b352-bd4f012dd9e7 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:39:43.067673 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb669d7-c8df-4b1c-b352-bd4f012dd9e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:39:43.134721 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-8cb669d7-c8df-4b1c-b352-bd4f012dd9e7 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:39:43.141562 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-8cb669d7-c8df-4b1c-b352-bd4f012dd9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:39:43.205617 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:39:43.251481 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 658/1324] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:39:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:39:43.288450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a29893-b666-48ce-a962-86956c42c14b None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:39:43.289946 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-74a29893-b666-48ce-a962-86956c42c14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:39:43.402878 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a29893-b666-48ce-a962-86956c42c14b None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:39:43.417602 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:39:43.494824 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b553fc03-ca68-421d-8373-aa7aecfcde68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:39:43.495692 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553fc03-ca68-421d-8373-aa7aecfcde68 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:39:43.495977 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553fc03-ca68-421d-8373-aa7aecfcde68 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:39:43.496258 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553fc03-ca68-421d-8373-aa7aecfcde68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:39:43.497919 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-b553fc03-ca68-421d-8373-aa7aecfcde68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:39:43.606899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b553fc03-ca68-421d-8373-aa7aecfcde68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:39:43.608346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b553fc03-ca68-421d-8373-aa7aecfcde68 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:39:43.626623 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 667/1325] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:39:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:39:45.695544 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3868e-e69a-43fa-8c2c-73850a956e19 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:39:45.696069 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3868e-e69a-43fa-8c2c-73850a956e19 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:39:45.696069 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3868e-e69a-43fa-8c2c-73850a956e19 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:39:45.697441 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 659/1326] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:39:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:39:45.702607 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd2eff-80c0-438f-8aa4-301e3b647e18 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:39:45.703015 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd2eff-80c0-438f-8aa4-301e3b647e18 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:39:45.703148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd2eff-80c0-438f-8aa4-301e3b647e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:39:45.751091 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:39:45.771329 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-a9bd2eff-80c0-438f-8aa4-301e3b647e18 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:39:45.772539 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-a9bd2eff-80c0-438f-8aa4-301e3b647e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:39:45.871607 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 668/1327] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:39:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:39:45.907022 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf3868e-e69a-43fa-8c2c-73850a956e19 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:39:45.908628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-ebf3868e-e69a-43fa-8c2c-73850a956e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:39:46.017454 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf3868e-e69a-43fa-8c2c-73850a956e19 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:39:46.093257 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a955d-7421-433d-9489-a76a46727fcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:39:46.093892 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a955d-7421-433d-9489-a76a46727fcb service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:39:46.094173 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a955d-7421-433d-9489-a76a46727fcb service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:39:46.094448 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a955d-7421-433d-9489-a76a46727fcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:39:46.096045 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-115a955d-7421-433d-9489-a76a46727fcb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:39:46.114304 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:39:46.223051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a955d-7421-433d-9489-a76a46727fcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:39:46.224494 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a955d-7421-433d-9489-a76a46727fcb service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:39:46.243944 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 660/1328] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:39:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:39:48.407679 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b3bc2-7e2f-4a21-963c-aaed91992e2e None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:39:48.408274 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b3bc2-7e2f-4a21-963c-aaed91992e2e None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:39:48.408274 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b3bc2-7e2f-4a21-963c-aaed91992e2e None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:39:48.409613 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 669/1329] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:39:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:39:48.415294 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf9276-2f1d-4e49-bc6d-89ceac15fb06 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:39:48.415838 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf9276-2f1d-4e49-bc6d-89ceac15fb06 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:39:48.416259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf9276-2f1d-4e49-bc6d-89ceac15fb06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:39:48.493950 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-d7bf9276-2f1d-4e49-bc6d-89ceac15fb06 None None] MFA Rules not processed for user `6e08ec69331f4a57817e3de09ab9221e`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:39:48.495827 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-d7bf9276-2f1d-4e49-bc6d-89ceac15fb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:39:48.579087 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:39:48.619851 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 661/1330] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:39:48 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:39:48.661553 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7b3bc2-7e2f-4a21-963c-aaed91992e2e None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:39:48.663118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-ea7b3bc2-7e2f-4a21-963c-aaed91992e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:39:48.793859 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7b3bc2-7e2f-4a21-963c-aaed91992e2e None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:39:48.818641 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:39:48.885747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d70a182-48eb-423b-8269-d43a4cc1a40f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:39:48.886779 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70a182-48eb-423b-8269-d43a4cc1a40f service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:39:48.887251 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70a182-48eb-423b-8269-d43a4cc1a40f service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:39:48.887706 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70a182-48eb-423b-8269-d43a4cc1a40f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:39:48.890387 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-4d70a182-48eb-423b-8269-d43a4cc1a40f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:39:49.021121 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d70a182-48eb-423b-8269-d43a4cc1a40f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:39:49.022857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d70a182-48eb-423b-8269-d43a4cc1a40f service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:39:49.044695 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 670/1331] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:39:48 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 22:39:51.376908 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1028649-e96c-4b9e-b49a-3ae03e21ede7 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:39:51.377525 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1028649-e96c-4b9e-b49a-3ae03e21ede7 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:39:51.377525 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1028649-e96c-4b9e-b49a-3ae03e21ede7 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:39:51.378837 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 662/1332] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:39:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:39:51.384110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75783b3-b303-4e50-be0f-df8d0f567a4b None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:39:51.384519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75783b3-b303-4e50-be0f-df8d0f567a4b None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:39:51.384654 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75783b3-b303-4e50-be0f-df8d0f567a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:39:51.449304 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:39:51.452347 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-d75783b3-b303-4e50-be0f-df8d0f567a4b None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:39:51.454197 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-d75783b3-b303-4e50-be0f-df8d0f567a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:39:51.552429 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 671/1333] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:39:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:39:51.588925 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1028649-e96c-4b9e-b49a-3ae03e21ede7 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:39:51.590490 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-b1028649-e96c-4b9e-b49a-3ae03e21ede7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:39:51.694406 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1028649-e96c-4b9e-b49a-3ae03e21ede7 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:39:51.767019 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f8034d-5a0c-4c6d-9d9f-a9eae25ed4a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:39:51.767623 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8034d-5a0c-4c6d-9d9f-a9eae25ed4a7 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:39:51.767901 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8034d-5a0c-4c6d-9d9f-a9eae25ed4a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:39:51.768170 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8034d-5a0c-4c6d-9d9f-a9eae25ed4a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:39:51.769780 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-f2f8034d-5a0c-4c6d-9d9f-a9eae25ed4a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:39:51.796843 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:39:51.878140 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f8034d-5a0c-4c6d-9d9f-a9eae25ed4a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:39:51.879591 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f8034d-5a0c-4c6d-9d9f-a9eae25ed4a7 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:39:51.897997 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 663/1334] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:39:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:39:53.968266 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb3c2b-9b88-431e-93ad-d9a4827570c0 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:39:53.968841 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb3c2b-9b88-431e-93ad-d9a4827570c0 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:39:53.968841 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb3c2b-9b88-431e-93ad-d9a4827570c0 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:39:53.970205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 672/1335] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:39:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:39:53.975650 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73d71e-4053-4584-8b68-62edda3a5e8a None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:39:53.976042 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73d71e-4053-4584-8b68-62edda3a5e8a None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:39:53.976200 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73d71e-4053-4584-8b68-62edda3a5e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:39:54.042849 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-6e73d71e-4053-4584-8b68-62edda3a5e8a None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:39:54.044088 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-6e73d71e-4053-4584-8b68-62edda3a5e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:39:54.123522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:39:54.147985 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 664/1336] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:39:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:39:54.188304 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb3c2b-9b88-431e-93ad-d9a4827570c0 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:39:54.189952 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-42fb3c2b-9b88-431e-93ad-d9a4827570c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:39:54.293721 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb3c2b-9b88-431e-93ad-d9a4827570c0 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:39:54.321806 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:39:54.365930 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353dd70b-51cc-4860-be7d-08889e7901cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:39:54.366531 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353dd70b-51cc-4860-be7d-08889e7901cf service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:39:54.366811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353dd70b-51cc-4860-be7d-08889e7901cf service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:39:54.367079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353dd70b-51cc-4860-be7d-08889e7901cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:39:54.369473 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-353dd70b-51cc-4860-be7d-08889e7901cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:39:54.475390 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353dd70b-51cc-4860-be7d-08889e7901cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:39:54.477380 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353dd70b-51cc-4860-be7d-08889e7901cf service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:39:54.495679 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 673/1337] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:39:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 22:39:56.824722 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe60967-b431-4241-9679-bb296526fd9f None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:39:56.825380 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe60967-b431-4241-9679-bb296526fd9f None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:39:56.825380 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe60967-b431-4241-9679-bb296526fd9f None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:39:56.826529 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 665/1338] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:39:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:39:56.831615 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e0791-8839-4bd4-ae0d-434461da3033 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:39:56.832010 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e0791-8839-4bd4-ae0d-434461da3033 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:39:56.832165 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e0791-8839-4bd4-ae0d-434461da3033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:39:56.897033 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-d37e0791-8839-4bd4-ae0d-434461da3033 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:39:56.898282 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-d37e0791-8839-4bd4-ae0d-434461da3033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:39:56.906977 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:39:56.999007 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 674/1339] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:39:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:39:57.035513 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe60967-b431-4241-9679-bb296526fd9f None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:39:57.037053 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-1fe60967-b431-4241-9679-bb296526fd9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:39:57.143152 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe60967-b431-4241-9679-bb296526fd9f None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:39:57.215331 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9670f2-8fcf-4894-a21d-a714b08b9cf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:39:57.215927 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9670f2-8fcf-4894-a21d-a714b08b9cf2 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:39:57.216204 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9670f2-8fcf-4894-a21d-a714b08b9cf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:39:57.216470 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9670f2-8fcf-4894-a21d-a714b08b9cf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:39:57.218079 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-2f9670f2-8fcf-4894-a21d-a714b08b9cf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:39:57.253918 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:39:57.330493 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9670f2-8fcf-4894-a21d-a714b08b9cf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:39:57.331921 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9670f2-8fcf-4894-a21d-a714b08b9cf2 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:39:57.349394 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 666/1340] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:39:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:40:01.359938 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe235f-8880-4505-828a-e7de339bc883 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:01.360496 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe235f-8880-4505-828a-e7de339bc883 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:01.360496 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe235f-8880-4505-828a-e7de339bc883 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:01.365982 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 675/1341] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:40:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:40:01.368921 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e84f5-7bca-4f7a-bad4-48f397971795 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:01.369402 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e84f5-7bca-4f7a-bad4-48f397971795 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:01.369512 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e84f5-7bca-4f7a-bad4-48f397971795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:01.440014 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-1f5e84f5-7bca-4f7a-bad4-48f397971795 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:40:01.441223 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-1f5e84f5-7bca-4f7a-bad4-48f397971795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:01.528879 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:01.540502 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 667/1342] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:40:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:40:01.580817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe235f-8880-4505-828a-e7de339bc883 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:40:01.582586 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-b6fe235f-8880-4505-828a-e7de339bc883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:01.692876 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe235f-8880-4505-828a-e7de339bc883 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:40:01.758424 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:01.798085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744fcea5-1573-4735-9e2c-4f8cc675cea8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:40:01.798616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fcea5-1573-4735-9e2c-4f8cc675cea8 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:01.798899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fcea5-1573-4735-9e2c-4f8cc675cea8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:01.799168 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fcea5-1573-4735-9e2c-4f8cc675cea8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:01.800766 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-744fcea5-1573-4735-9e2c-4f8cc675cea8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:01.911525 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744fcea5-1573-4735-9e2c-4f8cc675cea8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:40:01.912965 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744fcea5-1573-4735-9e2c-4f8cc675cea8 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:40:01.932937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 676/1343] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:40:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:40:04.594381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9119e3-9df0-455f-ad8a-c1fe45e9605c None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:04.594923 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9119e3-9df0-455f-ad8a-c1fe45e9605c None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:04.594923 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9119e3-9df0-455f-ad8a-c1fe45e9605c None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:04.596259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 668/1344] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:40:04.602940 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad612e24-40b6-402f-936a-a94c6b3470cf None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:04.603488 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad612e24-40b6-402f-936a-a94c6b3470cf None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:04.603488 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad612e24-40b6-402f-936a-a94c6b3470cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:04.676702 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-ad612e24-40b6-402f-936a-a94c6b3470cf None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:40:04.677985 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-ad612e24-40b6-402f-936a-a94c6b3470cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:04.698543 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:04.781595 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 677/1345] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:40:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:40:04.820653 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9119e3-9df0-455f-ad8a-c1fe45e9605c None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:40:04.822261 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-0e9119e3-9df0-455f-ad8a-c1fe45e9605c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:04.930422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9119e3-9df0-455f-ad8a-c1fe45e9605c None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:40:05.010585 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d63f7-b33b-4c0f-8cb1-86d98c01984d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:40:05.011238 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d63f7-b33b-4c0f-8cb1-86d98c01984d service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:05.011587 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d63f7-b33b-4c0f-8cb1-86d98c01984d service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:05.011924 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d63f7-b33b-4c0f-8cb1-86d98c01984d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:05.013975 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-854d63f7-b33b-4c0f-8cb1-86d98c01984d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:05.059948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:05.129804 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854d63f7-b33b-4c0f-8cb1-86d98c01984d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:40:05.131263 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854d63f7-b33b-4c0f-8cb1-86d98c01984d service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:40:05.151799 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 669/1346] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:40:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:40:09.572788 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad923ef8-ec06-47a7-9840-5f2fea4cd70f None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:09.573306 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad923ef8-ec06-47a7-9840-5f2fea4cd70f None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:09.573403 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad923ef8-ec06-47a7-9840-5f2fea4cd70f None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:09.574752 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 678/1347] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:40:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:40:09.580277 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a6551-32db-4c96-a81b-d42bb9c6a7a9 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:09.580775 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a6551-32db-4c96-a81b-d42bb9c6a7a9 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:09.580986 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a6551-32db-4c96-a81b-d42bb9c6a7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:09.654360 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-d45a6551-32db-4c96-a81b-d42bb9c6a7a9 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:40:09.655573 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-d45a6551-32db-4c96-a81b-d42bb9c6a7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:09.759638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:09.762317 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 670/1348] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:40:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:40:09.801238 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad923ef8-ec06-47a7-9840-5f2fea4cd70f None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:40:09.802813 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-ad923ef8-ec06-47a7-9840-5f2fea4cd70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:09.914600 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad923ef8-ec06-47a7-9840-5f2fea4cd70f None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:40:09.966802 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:09.995225 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10876c-5060-4e25-ab59-493e083a79b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:40:09.995845 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10876c-5060-4e25-ab59-493e083a79b7 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:09.996129 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10876c-5060-4e25-ab59-493e083a79b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:09.996404 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10876c-5060-4e25-ab59-493e083a79b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:09.998047 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-ac10876c-5060-4e25-ab59-493e083a79b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:10.234171 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac10876c-5060-4e25-ab59-493e083a79b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:40:10.235623 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac10876c-5060-4e25-ab59-493e083a79b7 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:40:10.254688 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 679/1349] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:40:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:40:15.200891 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d8ef3-484d-43b3-83d4-3259227ea171 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:15.201470 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d8ef3-484d-43b3-83d4-3259227ea171 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:15.201629 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d8ef3-484d-43b3-83d4-3259227ea171 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:15.205999 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 671/1350] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 10 22:40:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:40:15.211623 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9c528-6745-408b-a7bf-508a9ed05093 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:15.212055 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9c528-6745-408b-a7bf-508a9ed05093 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:15.212204 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9c528-6745-408b-a7bf-508a9ed05093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:15.328913 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-f7e9c528-6745-408b-a7bf-508a9ed05093 None None] MFA Rules not processed for user `9f46dee0f73e49eab54c11556e3b86bf`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:40:15.330450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-f7e9c528-6745-408b-a7bf-508a9ed05093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:15.364317 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:15.513160 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 680/1351] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 10 22:40:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:40:19.812165 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450c2d9-0f58-4b30-9ff9-0532287d80ad None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:19.812704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450c2d9-0f58-4b30-9ff9-0532287d80ad None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:19.812704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450c2d9-0f58-4b30-9ff9-0532287d80ad None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:19.815308 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 672/1352] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 10 22:40:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:40:19.820413 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a67c94-efe6-4bcd-a826-96a89b0dfb79 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:19.820896 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a67c94-efe6-4bcd-a826-96a89b0dfb79 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:19.820995 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a67c94-efe6-4bcd-a826-96a89b0dfb79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:19.900737 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-63a67c94-efe6-4bcd-a826-96a89b0dfb79 None None] MFA Rules not processed for user `9f46dee0f73e49eab54c11556e3b86bf`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:40:19.901969 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-63a67c94-efe6-4bcd-a826-96a89b0dfb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:19.976775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:20.017216 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 681/1353] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 10 22:40:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:40:21.396291 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d470a1-84e7-4c48-af72-42e17707d0a7 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:21.396881 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d470a1-84e7-4c48-af72-42e17707d0a7 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:21.396881 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d470a1-84e7-4c48-af72-42e17707d0a7 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:21.398189 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 673/1354] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 10 22:40:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:40:21.403284 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd976c-b13d-4da7-8015-b5bf562eeb39 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:21.403739 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd976c-b13d-4da7-8015-b5bf562eeb39 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:21.403835 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd976c-b13d-4da7-8015-b5bf562eeb39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:21.484117 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-b8cd976c-b13d-4da7-8015-b5bf562eeb39 None None] MFA Rules not processed for user `9f46dee0f73e49eab54c11556e3b86bf`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:40:21.485375 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-b8cd976c-b13d-4da7-8015-b5bf562eeb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:21.595706 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 682/1355] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 10 22:40:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:40:24.981603 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d7787-148d-4f9e-8c24-b2df2fba51ad None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:24.982425 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d7787-148d-4f9e-8c24-b2df2fba51ad None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:24.982704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d7787-148d-4f9e-8c24-b2df2fba51ad None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:24.986733 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 674/1356] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:40:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:40:24.991737 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2fe60-adfc-47e1-be6a-56360febd55a None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:24.993029 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2fe60-adfc-47e1-be6a-56360febd55a None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:24.993529 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2fe60-adfc-47e1-be6a-56360febd55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:25.031423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:25.093755 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-8eb2fe60-adfc-47e1-be6a-56360febd55a None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:40:25.095519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-8eb2fe60-adfc-47e1-be6a-56360febd55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:25.225325 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 683/1357] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:40:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:40:25.276164 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a7a76-971b-4aa5-99c5-9c915a326fe8 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:25.276907 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a7a76-971b-4aa5-99c5-9c915a326fe8 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:25.277629 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a7a76-971b-4aa5-99c5-9c915a326fe8 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:25.279638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 675/1358] 10.222.0.76 () {54 vars in 962 bytes} [Wed Aug 10 22:40:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:40:25.287819 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb068c3-6979-41a4-a817-e9683c69a0b1 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:25.288111 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb068c3-6979-41a4-a817-e9683c69a0b1 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:25.288390 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb068c3-6979-41a4-a817-e9683c69a0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:25.372177 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-cdb068c3-6979-41a4-a817-e9683c69a0b1 None None] MFA Rules not processed for user `b22838a8ad3949e59cdbaaa1801cd07f`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:40:25.373421 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-cdb068c3-6979-41a4-a817-e9683c69a0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:25.435440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:25.511852 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 684/1359] 10.222.0.76 () {58 vars in 1059 bytes} [Wed Aug 10 22:40:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:40:25.532013 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a7a76-971b-4aa5-99c5-9c915a326fe8 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:40:25.533886 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-051a7a76-971b-4aa5-99c5-9c915a326fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:25.701462 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a7a76-971b-4aa5-99c5-9c915a326fe8 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:40:25.780647 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:25.823578 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebebefc-5c30-4532-b044-acbcde082d99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:40:25.824189 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebebefc-5c30-4532-b044-acbcde082d99 service nova] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:25.824471 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebebefc-5c30-4532-b044-acbcde082d99 service nova] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:25.824746 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebebefc-5c30-4532-b044-acbcde082d99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:25.826418 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-6ebebefc-5c30-4532-b044-acbcde082d99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:25.922010 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebebefc-5c30-4532-b044-acbcde082d99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:40:25.924051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebebefc-5c30-4532-b044-acbcde082d99 service nova] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:40:25.944197 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 676/1360] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:40:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:40:28.203842 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03725c9b-9839-4132-8f06-57ad8170a81d None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:28.204367 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03725c9b-9839-4132-8f06-57ad8170a81d None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:28.204367 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03725c9b-9839-4132-8f06-57ad8170a81d None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:28.205913 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 685/1361] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:40:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:40:28.211038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24458616-73bd-4967-8ea7-a9740f51279a None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:28.211454 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24458616-73bd-4967-8ea7-a9740f51279a None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:28.211644 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24458616-73bd-4967-8ea7-a9740f51279a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:28.282513 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-24458616-73bd-4967-8ea7-a9740f51279a None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:40:28.283636 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-24458616-73bd-4967-8ea7-a9740f51279a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:28.337822 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:28.405554 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 677/1362] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:40:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:40:28.451130 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03725c9b-9839-4132-8f06-57ad8170a81d None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:40:28.452711 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-03725c9b-9839-4132-8f06-57ad8170a81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:28.635475 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03725c9b-9839-4132-8f06-57ad8170a81d None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:40:28.646762 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:28.794644 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2f1162-7ced-476a-afd0-e218b8f5c1bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:40:28.795285 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f1162-7ced-476a-afd0-e218b8f5c1bb service nova] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:28.795637 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f1162-7ced-476a-afd0-e218b8f5c1bb service nova] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:28.795978 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f1162-7ced-476a-afd0-e218b8f5c1bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:28.798079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-4d2f1162-7ced-476a-afd0-e218b8f5c1bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:28.894693 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2f1162-7ced-476a-afd0-e218b8f5c1bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:40:28.896304 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2f1162-7ced-476a-afd0-e218b8f5c1bb service nova] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:40:28.915876 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 686/1363] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:40:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:40:31.315442 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a95d5-c7f4-4db8-84d9-d8c1ec14b54c None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:31.316555 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a95d5-c7f4-4db8-84d9-d8c1ec14b54c None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:31.316555 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a95d5-c7f4-4db8-84d9-d8c1ec14b54c None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:31.318027 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 678/1364] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:40:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:40:31.323944 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c2f75-713d-4f4d-ad60-2c233f7ec321 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:31.324410 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c2f75-713d-4f4d-ad60-2c233f7ec321 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:31.324502 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c2f75-713d-4f4d-ad60-2c233f7ec321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:31.343317 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:31.432555 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-b89c2f75-713d-4f4d-ad60-2c233f7ec321 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:40:31.434640 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-b89c2f75-713d-4f4d-ad60-2c233f7ec321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:31.545271 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 687/1365] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:40:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:40:31.593526 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01b3a5-d259-47d5-b837-cc6deef550fd None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:31.594354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01b3a5-d259-47d5-b837-cc6deef550fd None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:31.594698 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01b3a5-d259-47d5-b837-cc6deef550fd None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:31.596184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 679/1366] 10.222.0.76 () {54 vars in 962 bytes} [Wed Aug 10 22:40:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:40:31.604531 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfc2f2-03b9-4862-88b1-4dfce6db2b99 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:31.604531 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfc2f2-03b9-4862-88b1-4dfce6db2b99 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:31.604531 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfc2f2-03b9-4862-88b1-4dfce6db2b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:31.689268 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-10cfc2f2-03b9-4862-88b1-4dfce6db2b99 None None] MFA Rules not processed for user `b22838a8ad3949e59cdbaaa1801cd07f`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:40:31.691580 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-10cfc2f2-03b9-4862-88b1-4dfce6db2b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:31.732093 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:31.825468 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 688/1367] 10.222.0.76 () {58 vars in 1059 bytes} [Wed Aug 10 22:40:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:40:31.830783 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01b3a5-d259-47d5-b837-cc6deef550fd None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:40:31.832525 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-1c01b3a5-d259-47d5-b837-cc6deef550fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:31.996263 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01b3a5-d259-47d5-b837-cc6deef550fd None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:40:32.020694 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:32.128900 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc3794-641e-403b-b3e7-de20634dc6e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:40:32.129542 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc3794-641e-403b-b3e7-de20634dc6e4 service nova] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:32.129827 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc3794-641e-403b-b3e7-de20634dc6e4 service nova] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:32.130104 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc3794-641e-403b-b3e7-de20634dc6e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:32.131716 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-24fc3794-641e-403b-b3e7-de20634dc6e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:32.227542 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc3794-641e-403b-b3e7-de20634dc6e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:40:32.229159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc3794-641e-403b-b3e7-de20634dc6e4 service nova] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:40:32.249223 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 680/1368] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:40:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 22:40:33.325906 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98497228-fec2-44f4-abff-dc9c1ffee27c None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:33.326420 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98497228-fec2-44f4-abff-dc9c1ffee27c None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:33.326420 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98497228-fec2-44f4-abff-dc9c1ffee27c None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:33.328013 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 689/1369] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:40:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:40:33.334257 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf479ca-b690-45b9-94f9-ce726b9b304e None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:33.334797 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf479ca-b690-45b9-94f9-ce726b9b304e None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:33.335029 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf479ca-b690-45b9-94f9-ce726b9b304e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:33.372418 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:33.420683 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-4bf479ca-b690-45b9-94f9-ce726b9b304e None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:40:33.421962 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-4bf479ca-b690-45b9-94f9-ce726b9b304e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:33.541317 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 681/1370] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:40:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:40:33.892498 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fbe481-898d-46ea-9b9a-4dc276ac69bf None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:33.892498 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fbe481-898d-46ea-9b9a-4dc276ac69bf None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:33.892498 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fbe481-898d-46ea-9b9a-4dc276ac69bf None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:33.892498 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 690/1371] 10.222.0.76 () {54 vars in 964 bytes} [Wed Aug 10 22:40:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:40:33.897409 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8abac-7272-4bb5-85ed-f4a31515118a None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:33.897698 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8abac-7272-4bb5-85ed-f4a31515118a None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:33.897972 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8abac-7272-4bb5-85ed-f4a31515118a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:33.983152 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-97c8abac-7272-4bb5-85ed-f4a31515118a None None] MFA Rules not processed for user `a7f0dd6c9e2d4315966e90cd3b927b13`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:40:33.984401 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-97c8abac-7272-4bb5-85ed-f4a31515118a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:34.038798 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:34.072800 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 682/1372] 10.222.0.76 () {58 vars in 1061 bytes} [Wed Aug 10 22:40:33 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:40:34.080647 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fbe481-898d-46ea-9b9a-4dc276ac69bf None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:40:34.082932 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-92fbe481-898d-46ea-9b9a-4dc276ac69bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:34.204368 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fbe481-898d-46ea-9b9a-4dc276ac69bf None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:40:34.247025 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:34.285887 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f6e60c-3d5c-4475-ab21-06e5e676d6f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:40:34.286496 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f6e60c-3d5c-4475-ab21-06e5e676d6f1 service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:34.286773 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f6e60c-3d5c-4475-ab21-06e5e676d6f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:34.287045 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f6e60c-3d5c-4475-ab21-06e5e676d6f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:34.288741 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-f1f6e60c-3d5c-4475-ab21-06e5e676d6f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:34.405356 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f6e60c-3d5c-4475-ab21-06e5e676d6f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:40:34.406284 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491bdc9d-d19e-40f1-8b08-456a7d18f798 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:34.406706 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491bdc9d-d19e-40f1-8b08-456a7d18f798 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:34.406796 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f6e60c-3d5c-4475-ab21-06e5e676d6f1 service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:40:34.406893 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491bdc9d-d19e-40f1-8b08-456a7d18f798 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:34.408200 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 683/1373] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:40:34.414437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdeaadc-205e-48da-a43e-ff29f74973b2 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:34.414842 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdeaadc-205e-48da-a43e-ff29f74973b2 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:34.415123 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdeaadc-205e-48da-a43e-ff29f74973b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:34.437496 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 691/1374] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:40:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:40:34.492300 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-3bdeaadc-205e-48da-a43e-ff29f74973b2 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:40:34.493578 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-3bdeaadc-205e-48da-a43e-ff29f74973b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:34.615594 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 684/1375] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:40:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:40:34.660653 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f6e60c-3d5c-4475-ab21-06e5e676d6f1 service cinder] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:40:34.661900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-f1f6e60c-3d5c-4475-ab21-06e5e676d6f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:34.738900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:34.782902 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f6e60c-3d5c-4475-ab21-06e5e676d6f1 service cinder] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:40:34.920540 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c402f318-efc8-42d6-bc78-e09a960df9a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:40:34.921083 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402f318-efc8-42d6-bc78-e09a960df9a8 service nova] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:34.921392 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402f318-efc8-42d6-bc78-e09a960df9a8 service nova] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:34.921669 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402f318-efc8-42d6-bc78-e09a960df9a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:34.923577 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-c402f318-efc8-42d6-bc78-e09a960df9a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:35.004832 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:35.017769 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c402f318-efc8-42d6-bc78-e09a960df9a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:40:35.019540 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c402f318-efc8-42d6-bc78-e09a960df9a8 service nova] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:40:35.040407 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 692/1376] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:40:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:40:37.124440 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1d825-6a0e-4570-a279-5b86342409bd None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:37.124917 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1d825-6a0e-4570-a279-5b86342409bd None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:37.124917 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1d825-6a0e-4570-a279-5b86342409bd None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:37.126288 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 685/1377] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:40:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:40:37.131711 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf61e91e-d55f-4f99-a3cb-1b4d778f4d92 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:37.132133 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf61e91e-d55f-4f99-a3cb-1b4d778f4d92 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:37.132244 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf61e91e-d55f-4f99-a3cb-1b4d778f4d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:37.204816 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-bf61e91e-d55f-4f99-a3cb-1b4d778f4d92 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:40:37.206106 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-bf61e91e-d55f-4f99-a3cb-1b4d778f4d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:37.306172 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 693/1378] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:40:37.351681 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1d825-6a0e-4570-a279-5b86342409bd None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:40:37.354996 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-f2d1d825-6a0e-4570-a279-5b86342409bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:37.419564 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:37.527613 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1d825-6a0e-4570-a279-5b86342409bd None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:40:37.659249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b8ad40-7c31-424a-9868-c60fae9cab52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:40:37.659872 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8ad40-7c31-424a-9868-c60fae9cab52 service nova] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:37.660144 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8ad40-7c31-424a-9868-c60fae9cab52 service nova] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:37.660408 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8ad40-7c31-424a-9868-c60fae9cab52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:37.662045 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-01b8ad40-7c31-424a-9868-c60fae9cab52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:37.674000 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:37.756636 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b8ad40-7c31-424a-9868-c60fae9cab52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:40:37.758293 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b8ad40-7c31-424a-9868-c60fae9cab52 service nova] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:40:37.776769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 686/1379] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:40:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 22:40:39.843722 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9070ba-c21a-498a-9df7-63b17465faa8 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:39.844253 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9070ba-c21a-498a-9df7-63b17465faa8 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:39.844253 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9070ba-c21a-498a-9df7-63b17465faa8 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:39.849821 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 694/1380] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:40:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:40:39.851292 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8b76c-85f8-4a8e-8bdd-da82ccb9dbbf None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:39.851702 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8b76c-85f8-4a8e-8bdd-da82ccb9dbbf None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:39.851831 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8b76c-85f8-4a8e-8bdd-da82ccb9dbbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:39.921918 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-b6d8b76c-85f8-4a8e-8bdd-da82ccb9dbbf None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:40:39.923158 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-b6d8b76c-85f8-4a8e-8bdd-da82ccb9dbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:40.014584 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:40.028904 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 687/1381] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:40:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:40:40.073868 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9070ba-c21a-498a-9df7-63b17465faa8 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:40:40.075407 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-ba9070ba-c21a-498a-9df7-63b17465faa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:40.151520 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:40.247423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9070ba-c21a-498a-9df7-63b17465faa8 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:40:40.376808 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acb8a9c-59ef-4d53-8efe-b10d522682a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:40:40.378036 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb8a9c-59ef-4d53-8efe-b10d522682a2 service nova] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:40.378376 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb8a9c-59ef-4d53-8efe-b10d522682a2 service nova] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:40.378717 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb8a9c-59ef-4d53-8efe-b10d522682a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:40.380380 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-9acb8a9c-59ef-4d53-8efe-b10d522682a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:40.406061 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:40.488382 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acb8a9c-59ef-4d53-8efe-b10d522682a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:40:40.490015 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acb8a9c-59ef-4d53-8efe-b10d522682a2 service nova] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:40:40.510392 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 695/1382] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:40:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:40:42.603077 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1b5d4-ea3c-4511-ac3a-6ed1cb6a66e6 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:42.603625 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1b5d4-ea3c-4511-ac3a-6ed1cb6a66e6 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:42.603854 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1b5d4-ea3c-4511-ac3a-6ed1cb6a66e6 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:42.605661 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 688/1383] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:40:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:40:42.612979 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69299eb0-9e95-45f9-935e-57574228313c None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:42.613504 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69299eb0-9e95-45f9-935e-57574228313c None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:42.613613 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69299eb0-9e95-45f9-935e-57574228313c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:42.683725 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-69299eb0-9e95-45f9-935e-57574228313c None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:40:42.685191 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-69299eb0-9e95-45f9-935e-57574228313c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:42.792329 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:42.797549 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 696/1384] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:40:42.840742 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1b5d4-ea3c-4511-ac3a-6ed1cb6a66e6 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:40:42.843873 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-01f1b5d4-ea3c-4511-ac3a-6ed1cb6a66e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:43.062103 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1b5d4-ea3c-4511-ac3a-6ed1cb6a66e6 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:40:43.148555 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:43.209317 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61233d-fc4c-4845-883e-4e406e3a852b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:40:43.210212 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61233d-fc4c-4845-883e-4e406e3a852b service nova] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:43.210667 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61233d-fc4c-4845-883e-4e406e3a852b service nova] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:43.210946 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61233d-fc4c-4845-883e-4e406e3a852b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:43.212605 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-0f61233d-fc4c-4845-883e-4e406e3a852b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:43.312880 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f61233d-fc4c-4845-883e-4e406e3a852b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:40:43.314642 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f61233d-fc4c-4845-883e-4e406e3a852b service nova] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:40:43.333244 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 689/1385] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:40:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:40:45.907378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b7c7c-5449-4e51-a019-901dac9d3d7f None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:45.907915 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b7c7c-5449-4e51-a019-901dac9d3d7f None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:45.907997 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b7c7c-5449-4e51-a019-901dac9d3d7f None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:45.909402 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 697/1386] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:40:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:40:45.915423 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008d895-62b5-4345-bc6c-86543e3265f8 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:45.915915 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008d895-62b5-4345-bc6c-86543e3265f8 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:45.915998 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008d895-62b5-4345-bc6c-86543e3265f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:45.995842 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-6008d895-62b5-4345-bc6c-86543e3265f8 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:40:45.997187 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-6008d895-62b5-4345-bc6c-86543e3265f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:46.029840 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:46.107766 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 690/1387] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:40:46.163008 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23b7c7c-5449-4e51-a019-901dac9d3d7f None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:40:46.164473 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-a23b7c7c-5449-4e51-a019-901dac9d3d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:46.329067 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23b7c7c-5449-4e51-a019-901dac9d3d7f None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:40:46.415521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:46.465035 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7f20c7-c46a-418d-958b-8a628018418a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:40:46.465660 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f20c7-c46a-418d-958b-8a628018418a service nova] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:46.466370 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f20c7-c46a-418d-958b-8a628018418a service nova] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:46.466370 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f20c7-c46a-418d-958b-8a628018418a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:46.467804 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-bd7f20c7-c46a-418d-958b-8a628018418a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:46.562647 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7f20c7-c46a-418d-958b-8a628018418a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:40:46.564417 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7f20c7-c46a-418d-958b-8a628018418a service nova] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:40:46.583292 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 698/1388] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:40:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:40:48.885676 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accf37d-e798-4dbb-99ae-b74667aaeaf1 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:48.886303 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accf37d-e798-4dbb-99ae-b74667aaeaf1 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:48.886303 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accf37d-e798-4dbb-99ae-b74667aaeaf1 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:48.887594 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 691/1389] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:40:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:40:48.892981 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a42f49-2e67-4204-b75e-2f56f7010161 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:48.893529 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a42f49-2e67-4204-b75e-2f56f7010161 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:48.893625 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a42f49-2e67-4204-b75e-2f56f7010161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:48.966082 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-a8a42f49-2e67-4204-b75e-2f56f7010161 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:40:48.967306 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-a8a42f49-2e67-4204-b75e-2f56f7010161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:49.003196 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:49.068206 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 699/1390] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:40:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:40:49.113471 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0accf37d-e798-4dbb-99ae-b74667aaeaf1 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:40:49.116156 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-0accf37d-e798-4dbb-99ae-b74667aaeaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:49.271312 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0accf37d-e798-4dbb-99ae-b74667aaeaf1 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:40:49.280843 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:49.387566 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02be7f80-a859-468b-b4de-667e31a191f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:40:49.388174 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02be7f80-a859-468b-b4de-667e31a191f4 service nova] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:49.388448 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02be7f80-a859-468b-b4de-667e31a191f4 service nova] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:49.388715 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02be7f80-a859-468b-b4de-667e31a191f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:49.390333 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-02be7f80-a859-468b-b4de-667e31a191f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:49.485927 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02be7f80-a859-468b-b4de-667e31a191f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:40:49.487541 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02be7f80-a859-468b-b4de-667e31a191f4 service nova] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:40:49.506373 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 692/1391] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:40:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:40:49.890686 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa0b29-60b8-4f37-b35a-3f5832027339 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:49.891604 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa0b29-60b8-4f37-b35a-3f5832027339 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:49.892105 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa0b29-60b8-4f37-b35a-3f5832027339 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:49.893517 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 700/1392] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:40:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:40:49.898854 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f42fc4f-342c-448b-9b92-932ef49aad3d None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:49.899312 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f42fc4f-342c-448b-9b92-932ef49aad3d None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:49.899443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f42fc4f-342c-448b-9b92-932ef49aad3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:49.915403 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:49.970105 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-6f42fc4f-342c-448b-9b92-932ef49aad3d None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:40:49.971333 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-6f42fc4f-342c-448b-9b92-932ef49aad3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:50.080981 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 693/1393] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:40:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:40:50.137634 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc411497-dc7c-436b-a5cc-5a099705fe79 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:50.137939 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc411497-dc7c-436b-a5cc-5a099705fe79 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:50.138215 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc411497-dc7c-436b-a5cc-5a099705fe79 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:50.139601 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 701/1394] 10.222.0.76 () {54 vars in 967 bytes} [Wed Aug 10 22:40:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:40:50.145694 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6a031-2ca2-4897-b9d0-e42131e2be44 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:50.146364 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6a031-2ca2-4897-b9d0-e42131e2be44 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:50.146364 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6a031-2ca2-4897-b9d0-e42131e2be44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:50.232895 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-b9c6a031-2ca2-4897-b9d0-e42131e2be44 None None] MFA Rules not processed for user `dc7524edf5d5422f91d92fd91be5ee1e`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:40:50.234155 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-b9c6a031-2ca2-4897-b9d0-e42131e2be44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:50.271252 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:50.315437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 694/1395] 10.222.0.76 () {58 vars in 1064 bytes} [Wed Aug 10 22:40:50 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:40:50.323067 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc411497-dc7c-436b-a5cc-5a099705fe79 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:40:50.324584 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-cc411497-dc7c-436b-a5cc-5a099705fe79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:50.434385 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc411497-dc7c-436b-a5cc-5a099705fe79 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:40:50.508167 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:50.521538 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80981c7-d169-46c1-a47c-ac13b138355b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc7524edf5d5422f91d92fd91be5ee1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:40:50.522133 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80981c7-d169-46c1-a47c-ac13b138355b service glance] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:50.522414 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80981c7-d169-46c1-a47c-ac13b138355b service glance] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:50.522687 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80981c7-d169-46c1-a47c-ac13b138355b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:50.524297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-b80981c7-d169-46c1-a47c-ac13b138355b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:50.648854 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80981c7-d169-46c1-a47c-ac13b138355b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:40:50.650326 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80981c7-d169-46c1-a47c-ac13b138355b service glance] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:40:50.675701 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 702/1396] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:40:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:40:51.502179 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b8709-3ae5-481a-8721-c7225065650f None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:51.502728 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b8709-3ae5-481a-8721-c7225065650f None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:51.502728 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b8709-3ae5-481a-8721-c7225065650f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:51.572763 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-5c1b8709-3ae5-481a-8721-c7225065650f None None] MFA Rules not processed for user `aff5e804c64345b298c745287a1d0a32`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:40:51.573994 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-5c1b8709-3ae5-481a-8721-c7225065650f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:51.663684 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 695/1397] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 10 22:40:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:40:51.710666 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8339581-4dc9-4361-a0cd-00b5fb6f587f None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:51.711194 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8339581-4dc9-4361-a0cd-00b5fb6f587f None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:51.711194 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8339581-4dc9-4361-a0cd-00b5fb6f587f None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:51.712572 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 703/1398] 10.222.0.76 () {54 vars in 943 bytes} [Wed Aug 10 22:40:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:40:51.719680 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e541a-b20f-40c1-9aa6-f554ffc59308 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:51.719895 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e541a-b20f-40c1-9aa6-f554ffc59308 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:51.720182 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e541a-b20f-40c1-9aa6-f554ffc59308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:51.729632 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8a797-9030-412f-b306-a674848369d7 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:51.729913 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8a797-9030-412f-b306-a674848369d7 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:51.730659 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8a797-9030-412f-b306-a674848369d7 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:51.731975 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 704/1399] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:40:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:40:51.738082 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc034e41-2487-4c33-915b-fcdfd783ab64 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:51.738380 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc034e41-2487-4c33-915b-fcdfd783ab64 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:51.738748 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc034e41-2487-4c33-915b-fcdfd783ab64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:51.782942 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:51.810066 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-6c6e541a-b20f-40c1-9aa6-f554ffc59308 None None] MFA Rules not processed for user `1e3a76df95a34b9fb9816f22b85d1071`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:40:51.811403 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-6c6e541a-b20f-40c1-9aa6-f554ffc59308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:51.815822 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-fc034e41-2487-4c33-915b-fcdfd783ab64 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:40:51.817010 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-fc034e41-2487-4c33-915b-fcdfd783ab64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:51.861405 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:51.917962 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 705/1400] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:40:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:40:51.925776 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 696/1401] 10.222.0.76 () {58 vars in 1040 bytes} [Wed Aug 10 22:40:51 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:40:51.935111 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc034e41-2487-4c33-915b-fcdfd783ab64 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:40:51.936319 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-fc034e41-2487-4c33-915b-fcdfd783ab64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:51.964492 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e541a-b20f-40c1-9aa6-f554ffc59308 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:40:51.966283 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-6c6e541a-b20f-40c1-9aa6-f554ffc59308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:52.043378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc034e41-2487-4c33-915b-fcdfd783ab64 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:40:52.069989 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e541a-b20f-40c1-9aa6-f554ffc59308 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:40:52.084800 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:52.110427 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:52.153481 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a70706c-036b-496d-a2db-cafb4cef8452 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e3a76df95a34b9fb9816f22b85d1071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:40:52.154527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70706c-036b-496d-a2db-cafb4cef8452 service swift] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:52.154806 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70706c-036b-496d-a2db-cafb4cef8452 service swift] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:52.155227 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70706c-036b-496d-a2db-cafb4cef8452 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:52.157610 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-2a70706c-036b-496d-a2db-cafb4cef8452 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:52.192936 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43faab6-1e41-47f9-8f4c-7e0273e8627d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:40:52.193514 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43faab6-1e41-47f9-8f4c-7e0273e8627d service nova] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:52.193792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43faab6-1e41-47f9-8f4c-7e0273e8627d service nova] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:52.194064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43faab6-1e41-47f9-8f4c-7e0273e8627d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:52.195679 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-b43faab6-1e41-47f9-8f4c-7e0273e8627d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:52.242234 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a70706c-036b-496d-a2db-cafb4cef8452 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:40:52.244009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a70706c-036b-496d-a2db-cafb4cef8452 service swift] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:40:52.252425 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 706/1402] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 10 22:40:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 22:40:52.291425 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43faab6-1e41-47f9-8f4c-7e0273e8627d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:40:52.293037 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43faab6-1e41-47f9-8f4c-7e0273e8627d service nova] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:40:52.313009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 697/1403] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:40:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 22:40:54.411356 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa807def-ae94-4a60-b340-30c1bba2c31e None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:54.411961 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa807def-ae94-4a60-b340-30c1bba2c31e None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:54.411961 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa807def-ae94-4a60-b340-30c1bba2c31e None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:54.419008 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 707/1404] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:40:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:40:54.419193 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a3b37-d1ed-4006-aa3e-51e5ba461a19 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:54.422067 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a3b37-d1ed-4006-aa3e-51e5ba461a19 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:54.422561 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a3b37-d1ed-4006-aa3e-51e5ba461a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:54.449171 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:54.508958 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-6f2a3b37-d1ed-4006-aa3e-51e5ba461a19 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:40:54.510196 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-6f2a3b37-d1ed-4006-aa3e-51e5ba461a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:54.621335 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 698/1405] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:40:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:40:54.663643 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa807def-ae94-4a60-b340-30c1bba2c31e None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:40:54.665101 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-fa807def-ae94-4a60-b340-30c1bba2c31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:54.779282 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:54.836459 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa807def-ae94-4a60-b340-30c1bba2c31e None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:40:54.967468 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053e5656-7348-485f-8f94-57b1b001e7e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:40:54.968953 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e5656-7348-485f-8f94-57b1b001e7e4 service nova] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:54.969443 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e5656-7348-485f-8f94-57b1b001e7e4 service nova] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:54.969878 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e5656-7348-485f-8f94-57b1b001e7e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:54.971657 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-053e5656-7348-485f-8f94-57b1b001e7e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:55.052290 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:55.076592 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053e5656-7348-485f-8f94-57b1b001e7e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:40:55.079033 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053e5656-7348-485f-8f94-57b1b001e7e4 service nova] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:40:55.099475 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 708/1406] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:40:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:40:57.160207 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4cfba-8f3b-4ca9-bb94-804cd4bce7f5 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:57.160778 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4cfba-8f3b-4ca9-bb94-804cd4bce7f5 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:57.160778 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4cfba-8f3b-4ca9-bb94-804cd4bce7f5 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:57.165355 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 699/1407] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:40:57.167648 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd375a2-0a73-46bb-8beb-2501c392de14 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:57.168119 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd375a2-0a73-46bb-8beb-2501c392de14 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:57.168209 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd375a2-0a73-46bb-8beb-2501c392de14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:57.245072 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-5cd375a2-0a73-46bb-8beb-2501c392de14 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:40:57.252687 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-5cd375a2-0a73-46bb-8beb-2501c392de14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:57.437392 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 709/1408] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:40:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:40:57.472487 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b4cfba-8f3b-4ca9-bb94-804cd4bce7f5 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:40:57.474028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-28b4cfba-8f3b-4ca9-bb94-804cd4bce7f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:57.578577 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:57.647961 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b4cfba-8f3b-4ca9-bb94-804cd4bce7f5 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:40:57.785686 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7787f66b-7eaf-4bfe-8007-ee8f58afdce8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:40:57.786319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7787f66b-7eaf-4bfe-8007-ee8f58afdce8 service nova] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:40:57.786604 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7787f66b-7eaf-4bfe-8007-ee8f58afdce8 service nova] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:40:57.786879 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7787f66b-7eaf-4bfe-8007-ee8f58afdce8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:40:57.788502 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-7787f66b-7eaf-4bfe-8007-ee8f58afdce8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:40:57.861630 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:40:57.898932 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7787f66b-7eaf-4bfe-8007-ee8f58afdce8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:40:57.900578 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7787f66b-7eaf-4bfe-8007-ee8f58afdce8 service nova] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:40:57.927413 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 700/1409] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:40:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:41:07.411548 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=81 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:07.924179 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=19 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:14.621241 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67bd27-9b38-4edd-86c2-300adb864270 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:14.621974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67bd27-9b38-4edd-86c2-300adb864270 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:14.621974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67bd27-9b38-4edd-86c2-300adb864270 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:14.623139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 710/1410] 10.222.0.76 () {54 vars in 918 bytes} [Wed Aug 10 22:41:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:41:14.628354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0738fe-c232-45d6-b2f2-abc0e4482d8d None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:14.628789 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0738fe-c232-45d6-b2f2-abc0e4482d8d None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:14.628912 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0738fe-c232-45d6-b2f2-abc0e4482d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:14.709844 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-ae0738fe-c232-45d6-b2f2-abc0e4482d8d None None] MFA Rules not processed for user `9f46dee0f73e49eab54c11556e3b86bf`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:41:14.710977 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-ae0738fe-c232-45d6-b2f2-abc0e4482d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:14.817322 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 701/1411] 10.222.0.76 () {58 vars in 1015 bytes} [Wed Aug 10 22:41:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:41:14.832650 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b81b17-90b0-4e26-a00b-6dbd3559fbcd None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:14.832933 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b81b17-90b0-4e26-a00b-6dbd3559fbcd None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:14.833209 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b81b17-90b0-4e26-a00b-6dbd3559fbcd None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:14.834676 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 711/1412] 10.222.0.76 () {54 vars in 918 bytes} [Wed Aug 10 22:41:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:41:14.839405 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299e4c9-c0fc-4f4e-9dc6-73ba2948f91d None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:14.839691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299e4c9-c0fc-4f4e-9dc6-73ba2948f91d None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:14.839959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299e4c9-c0fc-4f4e-9dc6-73ba2948f91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:14.912202 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:14.916047 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-4299e4c9-c0fc-4f4e-9dc6-73ba2948f91d None None] MFA Rules not processed for user `b22838a8ad3949e59cdbaaa1801cd07f`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:41:14.918085 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-4299e4c9-c0fc-4f4e-9dc6-73ba2948f91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:15.047220 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 702/1413] 10.222.0.76 () {58 vars in 1015 bytes} [Wed Aug 10 22:41:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:41:17.568654 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8984b8e-9d55-49a2-b849-0ff72441d886 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:17.569205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8984b8e-9d55-49a2-b849-0ff72441d886 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:17.569205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8984b8e-9d55-49a2-b849-0ff72441d886 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:17.570541 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 712/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 22:41:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:41:17.576879 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2bf576-01a7-42e1-86b0-462650e9385a None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:17.577339 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2bf576-01a7-42e1-86b0-462650e9385a None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:17.577494 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2bf576-01a7-42e1-86b0-462650e9385a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:17.653496 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-5d2bf576-01a7-42e1-86b0-462650e9385a None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:41:17.654734 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-5d2bf576-01a7-42e1-86b0-462650e9385a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:17.710480 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:17.753229 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 703/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 22:41:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:41:17.796749 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8984b8e-9d55-49a2-b849-0ff72441d886 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:41:17.798327 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-c8984b8e-9d55-49a2-b849-0ff72441d886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:17.944066 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8984b8e-9d55-49a2-b849-0ff72441d886 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:41:18.023752 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:18.065935 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1ee8c-cfa9-4596-ba22-7411740fadc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:41:18.066538 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1ee8c-cfa9-4596-ba22-7411740fadc7 service nova] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:18.066812 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1ee8c-cfa9-4596-ba22-7411740fadc7 service nova] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:18.067080 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1ee8c-cfa9-4596-ba22-7411740fadc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:18.068663 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-18f1ee8c-cfa9-4596-ba22-7411740fadc7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:18.155824 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f1ee8c-cfa9-4596-ba22-7411740fadc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:41:18.157542 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f1ee8c-cfa9-4596-ba22-7411740fadc7 service nova] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:41:18.176337 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 713/1416] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:41:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:41:19.080106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833fb6dd-9a00-4865-b672-daa385dd6093 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:19.080677 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833fb6dd-9a00-4865-b672-daa385dd6093 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:19.080677 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833fb6dd-9a00-4865-b672-daa385dd6093 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:19.082019 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 704/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 22:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:41:19.087070 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb718f-1865-4c86-a35b-5939cffc5f42 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:19.087530 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb718f-1865-4c86-a35b-5939cffc5f42 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:19.087639 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb718f-1865-4c86-a35b-5939cffc5f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:19.165151 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-65eb718f-1865-4c86-a35b-5939cffc5f42 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:41:19.166382 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-65eb718f-1865-4c86-a35b-5939cffc5f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:19.193526 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:19.262797 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 714/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 22:41:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:41:19.416784 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833fb6dd-9a00-4865-b672-daa385dd6093 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:41:19.419809 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-833fb6dd-9a00-4865-b672-daa385dd6093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:19.698322 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833fb6dd-9a00-4865-b672-daa385dd6093 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:41:19.738566 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:19.812195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c0199-5dfd-4978-9360-d7a524f35b07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:41:19.812827 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c0199-5dfd-4978-9360-d7a524f35b07 service nova] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:19.813099 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c0199-5dfd-4978-9360-d7a524f35b07 service nova] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:19.813395 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c0199-5dfd-4978-9360-d7a524f35b07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:19.814970 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-bc7c0199-5dfd-4978-9360-d7a524f35b07 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:19.902565 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c0199-5dfd-4978-9360-d7a524f35b07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:41:19.904160 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c0199-5dfd-4978-9360-d7a524f35b07 service nova] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:41:19.922761 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 705/1419] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:41:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:41:20.808688 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb54acc-ab98-48e4-8e3c-c99c167a0ac4 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:20.809225 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb54acc-ab98-48e4-8e3c-c99c167a0ac4 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:20.809225 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb54acc-ab98-48e4-8e3c-c99c167a0ac4 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:20.810489 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 715/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 22:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:41:20.815576 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9d2ad-8be1-45ca-9943-b225ca4b95c7 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:20.816020 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9d2ad-8be1-45ca-9943-b225ca4b95c7 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:20.816126 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9d2ad-8be1-45ca-9943-b225ca4b95c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:20.870314 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:20.893259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-16d9d2ad-8be1-45ca-9943-b225ca4b95c7 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:41:20.894497 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-16d9d2ad-8be1-45ca-9943-b225ca4b95c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:20.989786 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 706/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 22:41:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:41:21.152209 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb54acc-ab98-48e4-8e3c-c99c167a0ac4 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:41:21.153749 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-9cb54acc-ab98-48e4-8e3c-c99c167a0ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:21.296970 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb54acc-ab98-48e4-8e3c-c99c167a0ac4 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:41:21.309408 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:21.423109 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4493f7b-f848-4daf-b079-45a46716197b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:41:21.423639 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4493f7b-f848-4daf-b079-45a46716197b service nova] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:21.423921 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4493f7b-f848-4daf-b079-45a46716197b service nova] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:21.424196 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4493f7b-f848-4daf-b079-45a46716197b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:21.425870 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-b4493f7b-f848-4daf-b079-45a46716197b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:21.519436 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4493f7b-f848-4daf-b079-45a46716197b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:41:21.521032 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4493f7b-f848-4daf-b079-45a46716197b service nova] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:41:21.540058 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 716/1422] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:41:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:41:22.404986 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de953d5d-2dc2-4a07-9ad4-293f0d9702d2 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:22.405621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de953d5d-2dc2-4a07-9ad4-293f0d9702d2 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:22.405721 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de953d5d-2dc2-4a07-9ad4-293f0d9702d2 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:22.407285 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 707/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 22:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:41:22.412186 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378227fa-d08b-4e12-a621-497ed4d783ed None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:22.412595 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378227fa-d08b-4e12-a621-497ed4d783ed None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:22.412715 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378227fa-d08b-4e12-a621-497ed4d783ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:22.435487 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:22.491962 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-378227fa-d08b-4e12-a621-497ed4d783ed None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:41:22.493172 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-378227fa-d08b-4e12-a621-497ed4d783ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:22.590013 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 717/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 22:41:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:41:22.742828 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de953d5d-2dc2-4a07-9ad4-293f0d9702d2 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:41:22.745018 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-de953d5d-2dc2-4a07-9ad4-293f0d9702d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:22.857525 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:22.893461 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de953d5d-2dc2-4a07-9ad4-293f0d9702d2 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:41:23.005674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49afa446-cb2f-4aae-9606-a42a489d4d77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:41:23.006200 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49afa446-cb2f-4aae-9606-a42a489d4d77 service nova] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:23.006480 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49afa446-cb2f-4aae-9606-a42a489d4d77 service nova] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:23.006753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49afa446-cb2f-4aae-9606-a42a489d4d77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:23.008352 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-49afa446-cb2f-4aae-9606-a42a489d4d77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:23.084456 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:23.098682 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49afa446-cb2f-4aae-9606-a42a489d4d77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:41:23.100359 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49afa446-cb2f-4aae-9606-a42a489d4d77 service nova] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:41:23.119901 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 708/1425] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:41:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:41:23.984435 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5951f-f165-452b-97cf-c19cdd003aac None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:23.985123 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5951f-f165-452b-97cf-c19cdd003aac None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:23.985123 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5951f-f165-452b-97cf-c19cdd003aac None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:23.986445 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 718/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 22:41:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:41:23.991292 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb3877-3a32-419e-814f-e79cf701ef42 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:23.991720 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb3877-3a32-419e-814f-e79cf701ef42 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:23.991826 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb3877-3a32-419e-814f-e79cf701ef42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:24.072662 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-34eb3877-3a32-419e-814f-e79cf701ef42 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:41:24.073955 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-34eb3877-3a32-419e-814f-e79cf701ef42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:24.172266 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 709/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 22:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:41:24.322378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a5951f-f165-452b-97cf-c19cdd003aac None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:41:24.323811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-08a5951f-f165-452b-97cf-c19cdd003aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:24.407989 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:24.472685 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a5951f-f165-452b-97cf-c19cdd003aac None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:41:24.586585 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3656f424-ca5f-41a7-9193-16e26347aa8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:41:24.587761 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656f424-ca5f-41a7-9193-16e26347aa8d service nova] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:24.588250 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656f424-ca5f-41a7-9193-16e26347aa8d service nova] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:24.588614 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656f424-ca5f-41a7-9193-16e26347aa8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:24.590314 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-3656f424-ca5f-41a7-9193-16e26347aa8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:24.638979 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:24.680718 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3656f424-ca5f-41a7-9193-16e26347aa8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:41:24.682478 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3656f424-ca5f-41a7-9193-16e26347aa8d service nova] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:41:24.700532 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 719/1428] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:41:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:41:25.525465 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3577333-fa80-472d-9627-89f170777490 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:25.526037 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3577333-fa80-472d-9627-89f170777490 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:25.526142 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3577333-fa80-472d-9627-89f170777490 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:25.527496 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 710/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 22:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:41:25.532431 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e2e5a-f7ac-4d18-8a66-3e4100c8cc2b None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:25.532861 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e2e5a-f7ac-4d18-8a66-3e4100c8cc2b None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:25.532964 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e2e5a-f7ac-4d18-8a66-3e4100c8cc2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:25.612192 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-ae2e2e5a-f7ac-4d18-8a66-3e4100c8cc2b None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:41:25.613516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-ae2e2e5a-f7ac-4d18-8a66-3e4100c8cc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:25.712004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 720/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 22:41:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:41:25.864090 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3577333-fa80-472d-9627-89f170777490 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:41:25.865658 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-e3577333-fa80-472d-9627-89f170777490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:25.915767 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:26.012954 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3577333-fa80-472d-9627-89f170777490 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:41:26.125869 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb3699-45d5-42cb-a1f8-2e456c54c583 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:41:26.126477 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb3699-45d5-42cb-a1f8-2e456c54c583 service nova] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:26.126757 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb3699-45d5-42cb-a1f8-2e456c54c583 service nova] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:26.127031 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb3699-45d5-42cb-a1f8-2e456c54c583 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:26.128633 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-91bb3699-45d5-42cb-a1f8-2e456c54c583 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:26.136240 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:26.218238 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb3699-45d5-42cb-a1f8-2e456c54c583 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:41:26.219828 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb3699-45d5-42cb-a1f8-2e456c54c583 service nova] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:41:26.238300 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 711/1431] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:41:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:41:27.037727 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfaa4a0-727d-4e3f-a13a-7c095d552849 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:27.038590 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfaa4a0-727d-4e3f-a13a-7c095d552849 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:27.038590 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfaa4a0-727d-4e3f-a13a-7c095d552849 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:27.040450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 721/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 22:41:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:41:27.045409 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6d67e-5e8a-48eb-b638-f48d32a7c611 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:27.045986 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6d67e-5e8a-48eb-b638-f48d32a7c611 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:27.046266 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6d67e-5e8a-48eb-b638-f48d32a7c611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:27.124366 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-9ea6d67e-5e8a-48eb-b638-f48d32a7c611 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:41:27.125761 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-9ea6d67e-5e8a-48eb-b638-f48d32a7c611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:27.194200 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:27.222710 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 712/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 22:41:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:41:27.378006 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfaa4a0-727d-4e3f-a13a-7c095d552849 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:41:27.379514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-2cfaa4a0-727d-4e3f-a13a-7c095d552849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:27.396708 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:27.527256 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfaa4a0-727d-4e3f-a13a-7c095d552849 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:41:27.614129 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:27.640315 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc01512-3176-432d-a542-9d76c17bea1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:41:27.640852 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc01512-3176-432d-a542-9d76c17bea1e service nova] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:27.641126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc01512-3176-432d-a542-9d76c17bea1e service nova] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:27.641424 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc01512-3176-432d-a542-9d76c17bea1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:27.643003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-7cc01512-3176-432d-a542-9d76c17bea1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:27.732285 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc01512-3176-432d-a542-9d76c17bea1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:41:27.733908 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc01512-3176-432d-a542-9d76c17bea1e service nova] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:41:27.753954 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 722/1434] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:41:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 22:41:28.570917 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6728b-bd2d-4913-84fc-e307c31ae464 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:28.571476 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6728b-bd2d-4913-84fc-e307c31ae464 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:28.571717 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6728b-bd2d-4913-84fc-e307c31ae464 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:28.573078 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 713/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 22:41:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:41:28.578004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a4a68-1670-4c63-909a-de0dd64d1bcc None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:28.578402 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a4a68-1670-4c63-909a-de0dd64d1bcc None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:28.578554 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a4a68-1670-4c63-909a-de0dd64d1bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:28.656232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-d31a4a68-1670-4c63-909a-de0dd64d1bcc None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:41:28.657476 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-d31a4a68-1670-4c63-909a-de0dd64d1bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:28.700768 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:28.756100 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 723/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 22:41:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:41:28.909737 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b6728b-bd2d-4913-84fc-e307c31ae464 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:41:28.911242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-b8b6728b-bd2d-4913-84fc-e307c31ae464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:29.061724 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b6728b-bd2d-4913-84fc-e307c31ae464 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:41:29.119363 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:29.182654 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e955ed-0c75-41cb-9d58-f6230c8ae1d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:41:29.183260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e955ed-0c75-41cb-9d58-f6230c8ae1d2 service nova] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:29.183538 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e955ed-0c75-41cb-9d58-f6230c8ae1d2 service nova] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:29.183804 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e955ed-0c75-41cb-9d58-f6230c8ae1d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:29.185435 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-37e955ed-0c75-41cb-9d58-f6230c8ae1d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:29.276771 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e955ed-0c75-41cb-9d58-f6230c8ae1d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:41:29.278385 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e955ed-0c75-41cb-9d58-f6230c8ae1d2 service nova] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:41:29.296366 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 714/1437] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:41:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:41:30.106486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e50087-fe75-495d-af7f-1fedc4e70b53 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:30.107044 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e50087-fe75-495d-af7f-1fedc4e70b53 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:30.107152 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e50087-fe75-495d-af7f-1fedc4e70b53 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:30.108432 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 724/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 22:41:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:41:30.113521 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e292d57-ca63-4fd5-a42f-2aa782975a0e None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:30.113964 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e292d57-ca63-4fd5-a42f-2aa782975a0e None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:30.114157 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e292d57-ca63-4fd5-a42f-2aa782975a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:30.190304 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-0e292d57-ca63-4fd5-a42f-2aa782975a0e None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:41:30.191500 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-0e292d57-ca63-4fd5-a42f-2aa782975a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:30.193492 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:30.286882 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 715/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 22:41:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:41:30.445971 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e50087-fe75-495d-af7f-1fedc4e70b53 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:41:30.447559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-a7e50087-fe75-495d-af7f-1fedc4e70b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:30.592150 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e50087-fe75-495d-af7f-1fedc4e70b53 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:41:30.619222 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:30.727980 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b8c18-8320-49b7-9923-fc6d87abe5a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:41:30.728586 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b8c18-8320-49b7-9923-fc6d87abe5a3 service nova] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:30.728859 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b8c18-8320-49b7-9923-fc6d87abe5a3 service nova] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:30.729124 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b8c18-8320-49b7-9923-fc6d87abe5a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:30.730750 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-576b8c18-8320-49b7-9923-fc6d87abe5a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:30.820507 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b8c18-8320-49b7-9923-fc6d87abe5a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:41:30.822155 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b8c18-8320-49b7-9923-fc6d87abe5a3 service nova] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:41:30.839896 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 725/1440] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:41:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:41:32.751558 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eeb253-d023-449a-8334-57ce6004cf22 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:32.752093 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eeb253-d023-449a-8334-57ce6004cf22 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:32.752093 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eeb253-d023-449a-8334-57ce6004cf22 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:32.753373 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 716/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 22:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:41:32.758516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c4330-da7e-44c6-9252-bbe6d13e66e0 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:32.758953 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c4330-da7e-44c6-9252-bbe6d13e66e0 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:32.759049 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c4330-da7e-44c6-9252-bbe6d13e66e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:32.800368 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:32.826891 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-8c9c4330-da7e-44c6-9252-bbe6d13e66e0 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:41:32.828110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-8c9c4330-da7e-44c6-9252-bbe6d13e66e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:32.924461 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 726/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 22:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:41:32.964112 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eeb253-d023-449a-8334-57ce6004cf22 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:41:32.965680 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-00eeb253-d023-449a-8334-57ce6004cf22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:33.077714 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eeb253-d023-449a-8334-57ce6004cf22 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:41:33.110393 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:33.153196 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ef058-a4e2-4a67-adad-5570a5a39858 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:41:33.153835 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ef058-a4e2-4a67-adad-5570a5a39858 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:33.154114 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ef058-a4e2-4a67-adad-5570a5a39858 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:33.154386 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ef058-a4e2-4a67-adad-5570a5a39858 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:33.155980 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-687ef058-a4e2-4a67-adad-5570a5a39858 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:33.264414 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ef058-a4e2-4a67-adad-5570a5a39858 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:41:33.265883 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ef058-a4e2-4a67-adad-5570a5a39858 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:41:33.284054 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 717/1443] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:41:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:41:35.505503 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290e53c-2d66-4990-bc30-a8b96cb923a9 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:35.506051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290e53c-2d66-4990-bc30-a8b96cb923a9 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:35.506178 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290e53c-2d66-4990-bc30-a8b96cb923a9 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:35.508414 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 727/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 22:41:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:41:35.517027 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fe983-123e-497a-9d2d-7307c106764a None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:35.517728 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fe983-123e-497a-9d2d-7307c106764a None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:35.517728 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fe983-123e-497a-9d2d-7307c106764a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:35.584477 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-fe7fe983-123e-497a-9d2d-7307c106764a None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:41:35.585792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-fe7fe983-123e-497a-9d2d-7307c106764a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:35.598622 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:35.684546 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 718/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 22:41:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:41:35.723653 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290e53c-2d66-4990-bc30-a8b96cb923a9 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:41:35.725194 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-b290e53c-2d66-4990-bc30-a8b96cb923a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:35.828301 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290e53c-2d66-4990-bc30-a8b96cb923a9 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:41:35.840666 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:35.903001 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da3eaf-71e5-45b9-8e00-7f1b2ef256cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:41:35.903613 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da3eaf-71e5-45b9-8e00-7f1b2ef256cb service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:35.903892 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da3eaf-71e5-45b9-8e00-7f1b2ef256cb service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:35.904163 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da3eaf-71e5-45b9-8e00-7f1b2ef256cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:35.905799 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-91da3eaf-71e5-45b9-8e00-7f1b2ef256cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:36.019012 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da3eaf-71e5-45b9-8e00-7f1b2ef256cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:41:36.020460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da3eaf-71e5-45b9-8e00-7f1b2ef256cb service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:41:36.040012 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 728/1446] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:41:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:41:38.496145 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8993162-4c94-402c-afda-ed7d9a3132e8 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:38.496669 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8993162-4c94-402c-afda-ed7d9a3132e8 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:38.496784 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8993162-4c94-402c-afda-ed7d9a3132e8 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:38.498151 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 719/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 22:41:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:41:38.503519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a08061-66d9-4f4c-a684-0c0005468414 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:38.503965 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a08061-66d9-4f4c-a684-0c0005468414 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:38.504063 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a08061-66d9-4f4c-a684-0c0005468414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:38.552086 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:38.570836 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-73a08061-66d9-4f4c-a684-0c0005468414 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:41:38.572149 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-73a08061-66d9-4f4c-a684-0c0005468414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:38.669194 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 729/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 22:41:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:41:38.708948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8993162-4c94-402c-afda-ed7d9a3132e8 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:41:38.710923 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-c8993162-4c94-402c-afda-ed7d9a3132e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:38.816144 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8993162-4c94-402c-afda-ed7d9a3132e8 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:41:38.858439 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:38.892702 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf13aac-5862-40a1-8003-4afe15eac461 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:41:38.893337 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf13aac-5862-40a1-8003-4afe15eac461 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:38.893618 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf13aac-5862-40a1-8003-4afe15eac461 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:38.893885 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf13aac-5862-40a1-8003-4afe15eac461 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:38.895474 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-adf13aac-5862-40a1-8003-4afe15eac461 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:39.003653 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf13aac-5862-40a1-8003-4afe15eac461 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:41:39.005018 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf13aac-5862-40a1-8003-4afe15eac461 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:41:39.025013 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 720/1449] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:41:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:41:41.163161 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb9150-3509-41f8-9d4e-2b163829cd8e None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:41.163774 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb9150-3509-41f8-9d4e-2b163829cd8e None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:41.163774 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb9150-3509-41f8-9d4e-2b163829cd8e None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:41.165130 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 730/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 22:41:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:41:41.170592 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa268d9-5469-4899-aa3c-7469a7ceeffc None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:41.170997 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa268d9-5469-4899-aa3c-7469a7ceeffc None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:41.171143 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa268d9-5469-4899-aa3c-7469a7ceeffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:41.236824 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-6aa268d9-5469-4899-aa3c-7469a7ceeffc None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:41:41.238076 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-6aa268d9-5469-4899-aa3c-7469a7ceeffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:41.258947 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:41.335860 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 721/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 22:41:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:41:41.376438 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb9150-3509-41f8-9d4e-2b163829cd8e None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:41:41.378029 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-8bfb9150-3509-41f8-9d4e-2b163829cd8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:41.483886 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb9150-3509-41f8-9d4e-2b163829cd8e None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:41:41.503409 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:41.559279 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f311d9-6529-46e9-9d3a-1efcf92f184f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:41:41.559886 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f311d9-6529-46e9-9d3a-1efcf92f184f service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:41.560167 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f311d9-6529-46e9-9d3a-1efcf92f184f service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:41.560441 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f311d9-6529-46e9-9d3a-1efcf92f184f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:41.562088 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-97f311d9-6529-46e9-9d3a-1efcf92f184f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:41.674033 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f311d9-6529-46e9-9d3a-1efcf92f184f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:41:41.675379 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f311d9-6529-46e9-9d3a-1efcf92f184f service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:41:41.693414 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 731/1452] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:41:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:41:43.848217 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63a4f7-ccb8-4db6-8745-e081f962f230 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:43.848779 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63a4f7-ccb8-4db6-8745-e081f962f230 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:43.848779 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63a4f7-ccb8-4db6-8745-e081f962f230 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:43.850891 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 722/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 22:41:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:41:43.856357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d66b92-f217-4c96-990e-7cfde094bcb8 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:43.856781 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d66b92-f217-4c96-990e-7cfde094bcb8 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:43.856925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d66b92-f217-4c96-990e-7cfde094bcb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:43.920452 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:43.923794 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-b2d66b92-f217-4c96-990e-7cfde094bcb8 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:41:43.925545 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-b2d66b92-f217-4c96-990e-7cfde094bcb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:44.022884 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 732/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 22:41:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:41:44.061575 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f63a4f7-ccb8-4db6-8745-e081f962f230 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:41:44.063131 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-3f63a4f7-ccb8-4db6-8745-e081f962f230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:44.172501 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f63a4f7-ccb8-4db6-8745-e081f962f230 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:41:44.223501 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:44.249915 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0504f-46d7-4146-ab26-34bc80b4dc8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:41:44.250526 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0504f-46d7-4146-ab26-34bc80b4dc8d service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:44.250810 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0504f-46d7-4146-ab26-34bc80b4dc8d service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:44.251081 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0504f-46d7-4146-ab26-34bc80b4dc8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:44.252685 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-a2a0504f-46d7-4146-ab26-34bc80b4dc8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:44.361011 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a0504f-46d7-4146-ab26-34bc80b4dc8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:41:44.362489 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a0504f-46d7-4146-ab26-34bc80b4dc8d service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:41:44.383296 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 723/1455] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:41:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:41:46.517228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814f14c-86b9-4c65-aa8b-b12f54a88721 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:46.517795 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814f14c-86b9-4c65-aa8b-b12f54a88721 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:46.517795 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814f14c-86b9-4c65-aa8b-b12f54a88721 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:46.519138 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 733/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 22:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:41:46.525171 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a571cb9-68f5-4381-8aff-0e4404cf082b None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:46.525644 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a571cb9-68f5-4381-8aff-0e4404cf082b None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:46.525758 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a571cb9-68f5-4381-8aff-0e4404cf082b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:46.591703 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-1a571cb9-68f5-4381-8aff-0e4404cf082b None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:41:46.592915 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-1a571cb9-68f5-4381-8aff-0e4404cf082b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:46.622622 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:46.690289 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 724/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 22:41:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:41:46.729661 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814f14c-86b9-4c65-aa8b-b12f54a88721 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:41:46.731199 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-c814f14c-86b9-4c65-aa8b-b12f54a88721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:46.836069 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814f14c-86b9-4c65-aa8b-b12f54a88721 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:41:46.864241 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:46.909970 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6bbe56-7dc8-49a0-8468-972d3edd708e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:41:46.910577 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6bbe56-7dc8-49a0-8468-972d3edd708e service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:46.910857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6bbe56-7dc8-49a0-8468-972d3edd708e service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:46.911133 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6bbe56-7dc8-49a0-8468-972d3edd708e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:46.912766 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-7a6bbe56-7dc8-49a0-8468-972d3edd708e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:47.027262 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6bbe56-7dc8-49a0-8468-972d3edd708e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:41:47.028693 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6bbe56-7dc8-49a0-8468-972d3edd708e service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:41:47.047235 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 734/1458] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:41:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:41:49.151810 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c5e5a-edfe-4be5-9c49-417914811812 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:49.152431 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c5e5a-edfe-4be5-9c49-417914811812 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:49.152431 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c5e5a-edfe-4be5-9c49-417914811812 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:49.153714 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 725/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 22:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:41:49.159098 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6945004-9b74-4432-9ba8-9d0291798956 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:49.159542 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6945004-9b74-4432-9ba8-9d0291798956 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:49.159658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6945004-9b74-4432-9ba8-9d0291798956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:49.226090 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-d6945004-9b74-4432-9ba8-9d0291798956 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:41:49.227306 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-d6945004-9b74-4432-9ba8-9d0291798956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:49.235520 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:49.327375 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 735/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 22:41:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:41:49.365853 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c5e5a-edfe-4be5-9c49-417914811812 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:41:49.367400 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-cf3c5e5a-edfe-4be5-9c49-417914811812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:49.471376 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c5e5a-edfe-4be5-9c49-417914811812 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:41:49.530760 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:49.549718 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f716764b-3c11-4b5c-93f1-72829dedf22c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:41:49.550329 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716764b-3c11-4b5c-93f1-72829dedf22c service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:49.550610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716764b-3c11-4b5c-93f1-72829dedf22c service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:49.550880 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716764b-3c11-4b5c-93f1-72829dedf22c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:49.552866 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-f716764b-3c11-4b5c-93f1-72829dedf22c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:49.669388 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f716764b-3c11-4b5c-93f1-72829dedf22c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:41:49.670835 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f716764b-3c11-4b5c-93f1-72829dedf22c service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:41:49.689119 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 726/1461] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:41:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:41:51.795292 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e821684-dacb-45e7-b244-82d33dbc4a7b None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:51.795828 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e821684-dacb-45e7-b244-82d33dbc4a7b None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:51.795828 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e821684-dacb-45e7-b244-82d33dbc4a7b None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:51.797094 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 736/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 22:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:41:51.805403 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5eafb1-2b1f-4d9d-9df4-9af90bc205ad None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:51.805916 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5eafb1-2b1f-4d9d-9df4-9af90bc205ad None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:51.805916 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5eafb1-2b1f-4d9d-9df4-9af90bc205ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:51.883858 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-1e5eafb1-2b1f-4d9d-9df4-9af90bc205ad None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:41:51.885095 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-1e5eafb1-2b1f-4d9d-9df4-9af90bc205ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:51.935015 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:52.002576 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 727/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 22:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:41:52.048475 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e821684-dacb-45e7-b244-82d33dbc4a7b None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:41:52.050009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-7e821684-dacb-45e7-b244-82d33dbc4a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:52.176377 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e821684-dacb-45e7-b244-82d33dbc4a7b None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:41:52.228670 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:52.282466 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fcbab-19d5-4569-9300-5d3c1da83db0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:41:52.283325 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fcbab-19d5-4569-9300-5d3c1da83db0 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:52.283609 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fcbab-19d5-4569-9300-5d3c1da83db0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:52.283879 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fcbab-19d5-4569-9300-5d3c1da83db0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:52.285537 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-b59fcbab-19d5-4569-9300-5d3c1da83db0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:52.393530 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59fcbab-19d5-4569-9300-5d3c1da83db0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:41:52.395082 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59fcbab-19d5-4569-9300-5d3c1da83db0 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:41:52.413983 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 737/1464] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:41:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:41:54.488597 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8a959-4330-4673-be8d-3f242a30718f None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:54.489365 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8a959-4330-4673-be8d-3f242a30718f None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:54.489365 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8a959-4330-4673-be8d-3f242a30718f None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:54.490545 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 728/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 22:41:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:41:54.495994 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2281a6-0cd4-4159-8a82-65e27075e1b9 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:54.496417 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2281a6-0cd4-4159-8a82-65e27075e1b9 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:54.496546 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2281a6-0cd4-4159-8a82-65e27075e1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:54.563743 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-ca2281a6-0cd4-4159-8a82-65e27075e1b9 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:41:54.565019 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-ca2281a6-0cd4-4159-8a82-65e27075e1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:54.592237 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:54.678217 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 738/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 22:41:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:41:54.714711 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8a959-4330-4673-be8d-3f242a30718f None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:41:54.716307 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-95c8a959-4330-4673-be8d-3f242a30718f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:54.820537 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8a959-4330-4673-be8d-3f242a30718f None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:41:54.887192 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:54.894578 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70051b19-02ef-4030-80ec-ef236d35c8bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:41:54.895251 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70051b19-02ef-4030-80ec-ef236d35c8bd service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:54.895529 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70051b19-02ef-4030-80ec-ef236d35c8bd service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:54.895799 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70051b19-02ef-4030-80ec-ef236d35c8bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:54.897589 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-70051b19-02ef-4030-80ec-ef236d35c8bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:55.024516 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70051b19-02ef-4030-80ec-ef236d35c8bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:41:55.025928 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70051b19-02ef-4030-80ec-ef236d35c8bd service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:41:55.044867 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 729/1467] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:41:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:41:57.093704 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def8620-ecea-43a9-a350-2258483dcebc None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:57.094291 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def8620-ecea-43a9-a350-2258483dcebc None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:57.094291 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def8620-ecea-43a9-a350-2258483dcebc None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:57.095634 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 739/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 22:41:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:41:57.101787 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399ba1b-5872-49d6-b4f3-f91b8a6385dc None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:57.102249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399ba1b-5872-49d6-b4f3-f91b8a6385dc None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:57.102375 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399ba1b-5872-49d6-b4f3-f91b8a6385dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:57.169495 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-4399ba1b-5872-49d6-b4f3-f91b8a6385dc None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:41:57.170696 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-4399ba1b-5872-49d6-b4f3-f91b8a6385dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:57.219974 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:57.270531 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 730/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 22:41:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:41:57.307571 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def8620-ecea-43a9-a350-2258483dcebc None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:41:57.309104 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-1def8620-ecea-43a9-a350-2258483dcebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:57.415141 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def8620-ecea-43a9-a350-2258483dcebc None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:41:57.462386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:41:57.489166 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445542ba-2dd0-4b18-a6a1-81697cc6c18a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:41:57.489793 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445542ba-2dd0-4b18-a6a1-81697cc6c18a service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:41:57.490072 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445542ba-2dd0-4b18-a6a1-81697cc6c18a service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:41:57.490344 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445542ba-2dd0-4b18-a6a1-81697cc6c18a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:41:57.491929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-445542ba-2dd0-4b18-a6a1-81697cc6c18a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:41:57.600291 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445542ba-2dd0-4b18-a6a1-81697cc6c18a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:41:57.601754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445542ba-2dd0-4b18-a6a1-81697cc6c18a service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:41:57.620038 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 740/1470] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:41:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:42:07.267851 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=22 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:42:07.617853 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=66 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:42:17.228236 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800e94e-39d7-4845-879e-128264a398f9 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:42:17.228793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800e94e-39d7-4845-879e-128264a398f9 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:42:17.228793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800e94e-39d7-4845-879e-128264a398f9 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:42:17.230255 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 731/1471] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:42:17.236884 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f113e25-352c-4ef8-a001-53d1805b1b7d None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:42:17.237383 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f113e25-352c-4ef8-a001-53d1805b1b7d None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:42:17.237383 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f113e25-352c-4ef8-a001-53d1805b1b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:42:17.303653 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-1f113e25-352c-4ef8-a001-53d1805b1b7d None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:42:17.304866 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-1f113e25-352c-4ef8-a001-53d1805b1b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:42:17.407623 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 741/1472] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:42:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:42:17.490855 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c01624-fc0d-4f87-9678-60ede67cc11e None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:42:17.491081 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c01624-fc0d-4f87-9678-60ede67cc11e None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:42:17.491350 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c01624-fc0d-4f87-9678-60ede67cc11e None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:42:17.492676 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 732/1473] 10.222.0.76 () {54 vars in 967 bytes} [Wed Aug 10 22:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:42:17.497525 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16060f4b-4caa-4fcd-8947-1e0b41aaf319 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:42:17.497805 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16060f4b-4caa-4fcd-8947-1e0b41aaf319 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:42:17.498077 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16060f4b-4caa-4fcd-8947-1e0b41aaf319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:42:17.575086 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-16060f4b-4caa-4fcd-8947-1e0b41aaf319 None None] MFA Rules not processed for user `dc7524edf5d5422f91d92fd91be5ee1e`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:42:17.576307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-16060f4b-4caa-4fcd-8947-1e0b41aaf319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:42:17.598039 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:42:17.652959 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 742/1474] 10.222.0.76 () {58 vars in 1064 bytes} [Wed Aug 10 22:42:17 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:42:17.661398 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c01624-fc0d-4f87-9678-60ede67cc11e None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:42:17.662981 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-f3c01624-fc0d-4f87-9678-60ede67cc11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:42:17.771565 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c01624-fc0d-4f87-9678-60ede67cc11e None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:42:17.845018 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecdc98c-27d6-4397-bb82-f98de8555c08 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc7524edf5d5422f91d92fd91be5ee1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:42:17.845654 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecdc98c-27d6-4397-bb82-f98de8555c08 service glance] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:42:17.845934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecdc98c-27d6-4397-bb82-f98de8555c08 service glance] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:42:17.846205 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecdc98c-27d6-4397-bb82-f98de8555c08 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:42:17.847811 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-0ecdc98c-27d6-4397-bb82-f98de8555c08 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:42:17.901405 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:42:17.957529 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecdc98c-27d6-4397-bb82-f98de8555c08 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:42:17.958968 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecdc98c-27d6-4397-bb82-f98de8555c08 service glance] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:42:17.977910 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 733/1475] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:42:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:42:20.065670 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd5fb2-1dc7-499f-ac77-bcf9f5aca690 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:42:20.066259 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd5fb2-1dc7-499f-ac77-bcf9f5aca690 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:42:20.066259 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd5fb2-1dc7-499f-ac77-bcf9f5aca690 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:42:20.067595 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 743/1476] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:42:20.072782 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173abaee-5fec-44c4-9360-c31b7ce7737e None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:42:20.073195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173abaee-5fec-44c4-9360-c31b7ce7737e None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:42:20.073390 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173abaee-5fec-44c4-9360-c31b7ce7737e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:42:20.141698 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-173abaee-5fec-44c4-9360-c31b7ce7737e None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:42:20.142959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-173abaee-5fec-44c4-9360-c31b7ce7737e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:42:20.240756 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 734/1477] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:42:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:42:20.265140 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd5fb2-1dc7-499f-ac77-bcf9f5aca690 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:42:20.266722 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-a8cd5fb2-1dc7-499f-ac77-bcf9f5aca690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:42:20.394407 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd5fb2-1dc7-499f-ac77-bcf9f5aca690 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:42:20.445625 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:42:20.490544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e15f2b-d973-4998-bdae-cb6103dec42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:42:20.491151 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e15f2b-d973-4998-bdae-cb6103dec42c admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:42:20.491429 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e15f2b-d973-4998-bdae-cb6103dec42c admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:42:20.491697 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e15f2b-d973-4998-bdae-cb6103dec42c admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:42:20.492327 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e15f2b-d973-4998-bdae-cb6103dec42c admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:42:20.494088 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: WARNING py.warnings [None req-95e15f2b-d973-4998-bdae-cb6103dec42c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 22:42:20.494088 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: warnings.warn(msg) Aug 10 22:42:20.494088 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: Aug 10 22:42:20.495223 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e15f2b-d973-4998-bdae-cb6103dec42c admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:42:20.514855 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 744/1478] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 10 22:42:20 2022] GET /identity/v3/projects => generated 2553 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:42:22.550975 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c0071-ce45-410d-b765-a686ad76753f None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:42:22.551529 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c0071-ce45-410d-b765-a686ad76753f None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:42:22.551529 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c0071-ce45-410d-b765-a686ad76753f None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:42:22.552936 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 735/1479] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:42:22.558049 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e4636-e1a7-4987-938e-dc2cfcaf3f7a None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:42:22.558488 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e4636-e1a7-4987-938e-dc2cfcaf3f7a None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:42:22.558597 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e4636-e1a7-4987-938e-dc2cfcaf3f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:42:22.626575 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-e82e4636-e1a7-4987-938e-dc2cfcaf3f7a None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:42:22.627801 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-e82e4636-e1a7-4987-938e-dc2cfcaf3f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:42:22.735153 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 745/1480] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:42:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:42:22.777954 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c0071-ce45-410d-b765-a686ad76753f None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:42:22.779774 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-5f7c0071-ce45-410d-b765-a686ad76753f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:42:22.790102 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:42:22.923494 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c0071-ce45-410d-b765-a686ad76753f None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:42:23.014755 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:42:23.047914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e789e4-1f21-4aac-9cf3-418b48f4021d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:42:23.048529 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e789e4-1f21-4aac-9cf3-418b48f4021d service nova] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:42:23.048808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e789e4-1f21-4aac-9cf3-418b48f4021d service nova] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:42:23.049081 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e789e4-1f21-4aac-9cf3-418b48f4021d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:42:23.050721 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-c1e789e4-1f21-4aac-9cf3-418b48f4021d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:42:23.140546 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e789e4-1f21-4aac-9cf3-418b48f4021d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:42:23.142395 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e789e4-1f21-4aac-9cf3-418b48f4021d service nova] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:42:23.160552 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 736/1481] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:42:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 22:42:25.343295 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0566d-778c-4adb-8d66-bc96906b5091 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:42:25.343938 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0566d-778c-4adb-8d66-bc96906b5091 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:42:25.343938 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0566d-778c-4adb-8d66-bc96906b5091 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:42:25.346002 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 746/1482] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:42:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:42:25.351342 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6982baa-5668-4457-92ca-aba3b6ebabef None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:42:25.351764 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6982baa-5668-4457-92ca-aba3b6ebabef None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:42:25.351892 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6982baa-5668-4457-92ca-aba3b6ebabef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:42:25.420979 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-d6982baa-5668-4457-92ca-aba3b6ebabef None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:42:25.422150 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-d6982baa-5668-4457-92ca-aba3b6ebabef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:42:25.473021 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:42:25.525965 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 737/1483] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:42:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:42:25.563408 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a0566d-778c-4adb-8d66-bc96906b5091 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:42:25.564937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-77a0566d-778c-4adb-8d66-bc96906b5091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:42:25.588029 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:42:25.668621 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a0566d-778c-4adb-8d66-bc96906b5091 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:42:25.740916 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ee1bb-bb33-487e-a5af-1713cbb7ad17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:42:25.741482 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ee1bb-bb33-487e-a5af-1713cbb7ad17 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:42:25.741760 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ee1bb-bb33-487e-a5af-1713cbb7ad17 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:42:25.742029 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ee1bb-bb33-487e-a5af-1713cbb7ad17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:42:25.743619 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-051ee1bb-bb33-487e-a5af-1713cbb7ad17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:42:25.820962 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:42:25.856928 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051ee1bb-bb33-487e-a5af-1713cbb7ad17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:42:25.858408 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051ee1bb-bb33-487e-a5af-1713cbb7ad17 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:42:25.876602 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 747/1484] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:42:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:42:27.964731 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333c198-bcfb-4c64-9ccf-46e67fed89de None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:42:27.965369 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333c198-bcfb-4c64-9ccf-46e67fed89de None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:42:27.965369 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333c198-bcfb-4c64-9ccf-46e67fed89de None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:42:27.966669 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 738/1485] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:42:27.972075 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebd798-0f5a-4613-baf4-3b9bfd0e7b35 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:42:27.972497 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebd798-0f5a-4613-baf4-3b9bfd0e7b35 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:42:27.972635 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebd798-0f5a-4613-baf4-3b9bfd0e7b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:42:28.041902 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-10ebd798-0f5a-4613-baf4-3b9bfd0e7b35 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:42:28.043127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-10ebd798-0f5a-4613-baf4-3b9bfd0e7b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:42:28.142473 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 748/1486] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:42:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:42:28.182094 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6333c198-bcfb-4c64-9ccf-46e67fed89de None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:42:28.183636 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-6333c198-bcfb-4c64-9ccf-46e67fed89de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:42:28.288609 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6333c198-bcfb-4c64-9ccf-46e67fed89de None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:42:28.361049 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:42:28.363833 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9754e9-1473-4bb4-b335-b8a870d90b47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:42:28.364616 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9754e9-1473-4bb4-b335-b8a870d90b47 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:42:28.364895 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9754e9-1473-4bb4-b335-b8a870d90b47 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:42:28.365171 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9754e9-1473-4bb4-b335-b8a870d90b47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:42:28.367229 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-3f9754e9-1473-4bb4-b335-b8a870d90b47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:42:28.476042 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9754e9-1473-4bb4-b335-b8a870d90b47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:42:28.477519 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9754e9-1473-4bb4-b335-b8a870d90b47 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:42:28.496047 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 739/1487] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:42:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:42:30.641573 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f6483-4fad-4388-a2f3-c7a305b2593a None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:42:30.642161 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f6483-4fad-4388-a2f3-c7a305b2593a None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:42:30.642161 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f6483-4fad-4388-a2f3-c7a305b2593a None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:42:30.643466 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 749/1488] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:42:30.649551 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be28a6-675a-4119-991a-198b4d32067f None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:42:30.650029 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be28a6-675a-4119-991a-198b4d32067f None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:42:30.650135 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be28a6-675a-4119-991a-198b4d32067f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:42:30.717669 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-23be28a6-675a-4119-991a-198b4d32067f None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:42:30.718908 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-23be28a6-675a-4119-991a-198b4d32067f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:42:30.775383 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:42:30.821560 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 740/1489] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:42:30.861001 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954f6483-4fad-4388-a2f3-c7a305b2593a None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:42:30.862725 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-954f6483-4fad-4388-a2f3-c7a305b2593a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:42:30.900060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:42:30.968342 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954f6483-4fad-4388-a2f3-c7a305b2593a None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:42:31.043149 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e15ad1-abac-44cd-aae0-5ce0c7980392 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:42:31.043762 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e15ad1-abac-44cd-aae0-5ce0c7980392 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:42:31.044045 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e15ad1-abac-44cd-aae0-5ce0c7980392 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:42:31.044319 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e15ad1-abac-44cd-aae0-5ce0c7980392 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:42:31.045961 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-32e15ad1-abac-44cd-aae0-5ce0c7980392 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:42:31.128045 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:42:31.159176 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e15ad1-abac-44cd-aae0-5ce0c7980392 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:42:31.161664 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e15ad1-abac-44cd-aae0-5ce0c7980392 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:42:31.180724 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 750/1490] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:42:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:42:33.436850 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebefab8-a020-483d-afb9-8b011c5d8781 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:42:33.437254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebefab8-a020-483d-afb9-8b011c5d8781 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:42:33.437439 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebefab8-a020-483d-afb9-8b011c5d8781 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:42:33.438805 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 741/1491] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 10 22:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:42:33.444194 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a549f-2464-4eec-850f-f240327333a5 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:42:33.444699 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a549f-2464-4eec-850f-f240327333a5 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:42:33.444799 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a549f-2464-4eec-850f-f240327333a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:42:33.512294 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-b11a549f-2464-4eec-850f-f240327333a5 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:42:33.513557 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-b11a549f-2464-4eec-850f-f240327333a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:42:33.610436 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 751/1492] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 10 22:42:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:42:33.649949 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebefab8-a020-483d-afb9-8b011c5d8781 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:42:33.651531 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-1ebefab8-a020-483d-afb9-8b011c5d8781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:42:33.757063 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebefab8-a020-483d-afb9-8b011c5d8781 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:42:33.828739 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7898eb76-f4a7-4342-b680-40a10f55c91b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:42:33.829360 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898eb76-f4a7-4342-b680-40a10f55c91b service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:42:33.829637 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898eb76-f4a7-4342-b680-40a10f55c91b service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:42:33.829903 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898eb76-f4a7-4342-b680-40a10f55c91b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:42:33.831481 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-7898eb76-f4a7-4342-b680-40a10f55c91b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:42:33.842121 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:42:33.945677 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7898eb76-f4a7-4342-b680-40a10f55c91b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:42:33.947120 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7898eb76-f4a7-4342-b680-40a10f55c91b service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:42:33.965529 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 742/1493] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:42:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:42:43.609062 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=82 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:42:43.963208 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=50 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:43:52.818954 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65498956-0f5f-4f29-b335-4a6268dc6bdb None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:43:52.819621 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65498956-0f5f-4f29-b335-4a6268dc6bdb None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:43:52.819621 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65498956-0f5f-4f29-b335-4a6268dc6bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:43:52.910352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-65498956-0f5f-4f29-b335-4a6268dc6bdb None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:43:52.912061 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-65498956-0f5f-4f29-b335-4a6268dc6bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:43:53.027858 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 752/1494] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:43:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:43:53.036742 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f6037-a6fa-4702-aae3-635765266116 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:43:53.037272 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f6037-a6fa-4702-aae3-635765266116 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:43:53.037272 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f6037-a6fa-4702-aae3-635765266116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:43:53.113600 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-f36f6037-a6fa-4702-aae3-635765266116 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:43:53.114821 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-f36f6037-a6fa-4702-aae3-635765266116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:43:53.211531 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 743/1495] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:43:53.216344 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65498956-0f5f-4f29-b335-4a6268dc6bdb None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:43:53.217607 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-65498956-0f5f-4f29-b335-4a6268dc6bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:43:53.229365 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65498956-0f5f-4f29-b335-4a6268dc6bdb None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:43:53.291138 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:43:53.325196 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f7653-b0ca-4890-b712-037037567b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:43:53.326032 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f7653-b0ca-4890-b712-037037567b05 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:43:53.326318 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f7653-b0ca-4890-b712-037037567b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:43:53.326597 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f7653-b0ca-4890-b712-037037567b05 admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:43:53.327558 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151f7653-b0ca-4890-b712-037037567b05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:43:53.328054 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151f7653-b0ca-4890-b712-037037567b05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:43:53.329652 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: WARNING py.warnings [None req-151f7653-b0ca-4890-b712-037037567b05 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 22:43:53.329652 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: warnings.warn(msg) Aug 10 22:43:53.329652 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: Aug 10 22:43:53.331242 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151f7653-b0ca-4890-b712-037037567b05 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:43:53.343059 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 753/1496] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:43:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:43:53.351416 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f6037-a6fa-4702-aae3-635765266116 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:43:53.361260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f6037-a6fa-4702-aae3-635765266116 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:43:53.423934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:43:53.459947 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cda961-b2ec-4ace-afa8-ee7cff2753a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:43:53.460559 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cda961-b2ec-4ace-afa8-ee7cff2753a0 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:43:53.460833 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cda961-b2ec-4ace-afa8-ee7cff2753a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:43:53.461101 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cda961-b2ec-4ace-afa8-ee7cff2753a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:43:53.461750 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cda961-b2ec-4ace-afa8-ee7cff2753a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:43:53.463488 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: WARNING py.warnings [None req-16cda961-b2ec-4ace-afa8-ee7cff2753a0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 22:43:53.463488 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: warnings.warn(msg) Aug 10 22:43:53.463488 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: Aug 10 22:43:53.464678 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cda961-b2ec-4ace-afa8-ee7cff2753a0 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:43:53.520973 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 744/1497] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:43:53 2022] POST /identity/v3/projects => generated 377 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:43:53.525265 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f7653-b0ca-4890-b712-037037567b05 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:43:53.526838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-151f7653-b0ca-4890-b712-037037567b05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:43:53.560612 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f7653-b0ca-4890-b712-037037567b05 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:43:53.657199 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787cc85f-82bb-4acc-97d0-08183edf3220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:43:53.657838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787cc85f-82bb-4acc-97d0-08183edf3220 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:43:53.658114 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787cc85f-82bb-4acc-97d0-08183edf3220 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:43:53.658381 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787cc85f-82bb-4acc-97d0-08183edf3220 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:43:53.659006 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787cc85f-82bb-4acc-97d0-08183edf3220 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:43:53.660018 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: WARNING py.warnings [None req-787cc85f-82bb-4acc-97d0-08183edf3220 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 22:43:53.660018 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: warnings.warn(msg) Aug 10 22:43:53.660018 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: Aug 10 22:43:53.661216 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787cc85f-82bb-4acc-97d0-08183edf3220 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:43:53.702655 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 754/1498] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:43:53 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:43:53.706805 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cda961-b2ec-4ace-afa8-ee7cff2753a0 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:43:53.708308 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-16cda961-b2ec-4ace-afa8-ee7cff2753a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:43:53.771216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cda961-b2ec-4ace-afa8-ee7cff2753a0 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:43:53.854675 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:43:53.866524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d7680-d938-48fd-8e75-a849a1918893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:43:53.867135 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d7680-d938-48fd-8e75-a849a1918893 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:43:53.867412 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d7680-d938-48fd-8e75-a849a1918893 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:43:53.867682 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d7680-d938-48fd-8e75-a849a1918893 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:43:53.868320 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2d7680-d938-48fd-8e75-a849a1918893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:43:53.869385 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: WARNING py.warnings [None req-9a2d7680-d938-48fd-8e75-a849a1918893 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 22:43:53.869385 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: warnings.warn(msg) Aug 10 22:43:53.869385 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: Aug 10 22:43:53.870274 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2d7680-d938-48fd-8e75-a849a1918893 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:43:53.883282 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 745/1499] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:43:53.887601 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787cc85f-82bb-4acc-97d0-08183edf3220 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:43:53.889904 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-787cc85f-82bb-4acc-97d0-08183edf3220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:43:53.901455 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787cc85f-82bb-4acc-97d0-08183edf3220 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:43:53.928515 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:43:54.021616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738d1b0-66e3-4c6a-9d2d-19324e947d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:43:54.022151 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738d1b0-66e3-4c6a-9d2d-19324e947d23 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:43:54.022421 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738d1b0-66e3-4c6a-9d2d-19324e947d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:43:54.022687 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738d1b0-66e3-4c6a-9d2d-19324e947d23 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:43:54.023318 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5738d1b0-66e3-4c6a-9d2d-19324e947d23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:43:54.024334 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: WARNING py.warnings [None req-5738d1b0-66e3-4c6a-9d2d-19324e947d23 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 22:43:54.024334 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: warnings.warn(msg) Aug 10 22:43:54.024334 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: Aug 10 22:43:54.025208 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5738d1b0-66e3-4c6a-9d2d-19324e947d23 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:43:54.037935 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 755/1500] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:43:54.042402 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d7680-d938-48fd-8e75-a849a1918893 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:43:54.043931 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-9a2d7680-d938-48fd-8e75-a849a1918893 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:43:54.079023 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d7680-d938-48fd-8e75-a849a1918893 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:43:54.173264 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f3997-861f-4131-abb4-b76b55b01d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:43:54.173964 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f3997-861f-4131-abb4-b76b55b01d97 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:43:54.174239 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f3997-861f-4131-abb4-b76b55b01d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:43:54.174506 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f3997-861f-4131-abb4-b76b55b01d97 admin admin] PATH_INFO: `/v3/projects/8f0e9dc356ee4078967a3fa41e17a134/users/1fd04a6e782a4100ac9984f938515b01/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:43:54.196916 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f3997-861f-4131-abb4-b76b55b01d97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f0e9dc356ee4078967a3fa41e17a134, user_id=1fd04a6e782a4100ac9984f938515b01, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:43:54.197988 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: WARNING py.warnings [None req-ac5f3997-861f-4131-abb4-b76b55b01d97 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 22:43:54.197988 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: warnings.warn(msg) Aug 10 22:43:54.197988 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: Aug 10 22:43:54.199935 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f3997-861f-4131-abb4-b76b55b01d97 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:43:54.224769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 746/1501] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:43:54 2022] PUT /identity/v3/projects/8f0e9dc356ee4078967a3fa41e17a134/users/1fd04a6e782a4100ac9984f938515b01/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:43:54.237870 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd9d241-ae5a-406f-8d64-1ea2f34d1877 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:43:54.238155 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd9d241-ae5a-406f-8d64-1ea2f34d1877 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:43:54.238427 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd9d241-ae5a-406f-8d64-1ea2f34d1877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:43:54.308304 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-7fd9d241-ae5a-406f-8d64-1ea2f34d1877 None None] MFA Rules not processed for user `1fd04a6e782a4100ac9984f938515b01`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:43:54.309730 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-7fd9d241-ae5a-406f-8d64-1ea2f34d1877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:43:54.386126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 756/1502] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:43:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:43:54.399299 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c7a96-ec72-46d1-88b0-70d5d0c6eb22 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:43:54.399584 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c7a96-ec72-46d1-88b0-70d5d0c6eb22 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:43:54.399862 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c7a96-ec72-46d1-88b0-70d5d0c6eb22 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:43:54.401209 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 747/1503] 10.222.0.76 () {54 vars in 964 bytes} [Wed Aug 10 22:43:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:43:54.406315 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7b7af-c13d-4c14-9cdf-faad20c35255 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:43:54.406700 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7b7af-c13d-4c14-9cdf-faad20c35255 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:43:54.407065 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7b7af-c13d-4c14-9cdf-faad20c35255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:43:54.440815 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:43:54.484881 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-87d7b7af-c13d-4c14-9cdf-faad20c35255 None None] MFA Rules not processed for user `a7f0dd6c9e2d4315966e90cd3b927b13`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:43:54.486129 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-87d7b7af-c13d-4c14-9cdf-faad20c35255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:43:54.562075 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 757/1504] 10.222.0.76 () {58 vars in 1061 bytes} [Wed Aug 10 22:43:54 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:43:54.569902 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c7a96-ec72-46d1-88b0-70d5d0c6eb22 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:43:54.571419 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-4c9c7a96-ec72-46d1-88b0-70d5d0c6eb22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:43:54.621373 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:43:54.677265 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c7a96-ec72-46d1-88b0-70d5d0c6eb22 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:43:54.753457 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16633e06-6c95-478c-9af7-67045dad51bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:43:54.754065 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16633e06-6c95-478c-9af7-67045dad51bf service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:43:54.754346 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16633e06-6c95-478c-9af7-67045dad51bf service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:43:54.754618 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16633e06-6c95-478c-9af7-67045dad51bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:43:54.756220 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-16633e06-6c95-478c-9af7-67045dad51bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:43:54.857480 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16633e06-6c95-478c-9af7-67045dad51bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:43:54.859861 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16633e06-6c95-478c-9af7-67045dad51bf service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:43:54.863425 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:43:54.878475 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 748/1505] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:43:54 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:43:54.923771 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107d798-1ab3-4597-816d-5af17de604bc None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:43:54.924207 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107d798-1ab3-4597-816d-5af17de604bc None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:43:54.924346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107d798-1ab3-4597-816d-5af17de604bc None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:43:54.925619 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 758/1506] 10.222.0.76 () {52 vars in 830 bytes} [Wed Aug 10 22:43:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:43:54.929202 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16633e06-6c95-478c-9af7-67045dad51bf service cinder] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:43:54.930446 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-16633e06-6c95-478c-9af7-67045dad51bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:43:55.030520 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16633e06-6c95-478c-9af7-67045dad51bf service cinder] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:43:55.123961 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdebfc96-453c-4921-9331-f2dd3d30804b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:43:55.124617 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebfc96-453c-4921-9331-f2dd3d30804b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:43:55.124810 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebfc96-453c-4921-9331-f2dd3d30804b admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:43:55.125084 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebfc96-453c-4921-9331-f2dd3d30804b admin admin] PATH_INFO: `/v3/users/1fd04a6e782a4100ac9984f938515b01` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:43:55.137473 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdebfc96-453c-4921-9331-f2dd3d30804b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fd04a6e782a4100ac9984f938515b01)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:43:55.138530 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: WARNING py.warnings [None req-bdebfc96-453c-4921-9331-f2dd3d30804b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 22:43:55.138530 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: warnings.warn(msg) Aug 10 22:43:55.138530 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: Aug 10 22:43:55.140280 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdebfc96-453c-4921-9331-f2dd3d30804b admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:43:55.149780 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:43:55.209428 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-bdebfc96-453c-4921-9331-f2dd3d30804b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fd04a6e782a4100ac9984f938515b01'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:43:55.224681 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-bdebfc96-453c-4921-9331-f2dd3d30804b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fd04a6e782a4100ac9984f938515b01'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:43:55.247473 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-bdebfc96-453c-4921-9331-f2dd3d30804b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fd04a6e782a4100ac9984f938515b01'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:43:55.247929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-bdebfc96-453c-4921-9331-f2dd3d30804b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fd04a6e782a4100ac9984f938515b01'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:43:55.248459 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-bdebfc96-453c-4921-9331-f2dd3d30804b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fd04a6e782a4100ac9984f938515b01'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:43:55.263808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 749/1507] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:43:54 2022] DELETE /identity/v3/users/1fd04a6e782a4100ac9984f938515b01 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:43:55.268859 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40db23-f22e-408c-9a93-186be7e5a4c6 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:43:55.269148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40db23-f22e-408c-9a93-186be7e5a4c6 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:43:55.269460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40db23-f22e-408c-9a93-186be7e5a4c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:43:55.329523 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-0b40db23-f22e-408c-9a93-186be7e5a4c6 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:43:55.330746 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-0b40db23-f22e-408c-9a93-186be7e5a4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:43:55.407438 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:43:55.421188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 759/1508] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:43:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:43:55.431115 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdebfc96-453c-4921-9331-f2dd3d30804b admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:43:55.432229 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-bdebfc96-453c-4921-9331-f2dd3d30804b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:43:55.537763 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdebfc96-453c-4921-9331-f2dd3d30804b admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:43:55.593474 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:43:55.616298 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b029e683-746c-42ba-b573-354c65011093 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:43:55.616950 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029e683-746c-42ba-b573-354c65011093 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:43:55.617233 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029e683-746c-42ba-b573-354c65011093 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:43:55.617538 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029e683-746c-42ba-b573-354c65011093 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:43:55.619483 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-b029e683-746c-42ba-b573-354c65011093 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:43:55.729370 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b029e683-746c-42ba-b573-354c65011093 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:43:55.730759 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b029e683-746c-42ba-b573-354c65011093 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:43:55.750427 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 750/1509] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:43:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:43:56.130968 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40db23-f22e-408c-9a93-186be7e5a4c6 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:43:56.132242 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-0b40db23-f22e-408c-9a93-186be7e5a4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:43:56.144038 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40db23-f22e-408c-9a93-186be7e5a4c6 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:43:56.242670 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645d75f0-9d68-403c-91c5-526c151c9945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:43:56.243295 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d75f0-9d68-403c-91c5-526c151c9945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:43:56.243571 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d75f0-9d68-403c-91c5-526c151c9945 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:43:56.243841 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d75f0-9d68-403c-91c5-526c151c9945 admin admin] PATH_INFO: `/v3/projects/8f0e9dc356ee4078967a3fa41e17a134` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:43:56.254374 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645d75f0-9d68-403c-91c5-526c151c9945 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f0e9dc356ee4078967a3fa41e17a134)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:43:56.254594 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: WARNING py.warnings [None req-645d75f0-9d68-403c-91c5-526c151c9945 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 22:43:56.254594 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: warnings.warn(msg) Aug 10 22:43:56.254594 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: Aug 10 22:43:56.255826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645d75f0-9d68-403c-91c5-526c151c9945 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:43:56.315668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-645d75f0-9d68-403c-91c5-526c151c9945 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f0e9dc356ee4078967a3fa41e17a134', 'request_id': 'req-645d75f0-9d68-403c-91c5-526c151c9945'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:43:56.321738 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-645d75f0-9d68-403c-91c5-526c151c9945 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f0e9dc356ee4078967a3fa41e17a134', 'request_id': 'req-645d75f0-9d68-403c-91c5-526c151c9945'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:43:56.330446 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-645d75f0-9d68-403c-91c5-526c151c9945 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f0e9dc356ee4078967a3fa41e17a134', 'request_id': 'req-645d75f0-9d68-403c-91c5-526c151c9945'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:43:56.361616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-645d75f0-9d68-403c-91c5-526c151c9945 admin admin] The token cache is being invalidate because project 8f0e9dc356ee4078967a3fa41e17a134 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:43:56.361921 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-645d75f0-9d68-403c-91c5-526c151c9945 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:43:56.362472 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-645d75f0-9d68-403c-91c5-526c151c9945 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:43:56.364058 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 760/1510] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:43:56 2022] DELETE /identity/v3/projects/8f0e9dc356ee4078967a3fa41e17a134 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:44:05.747373 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=62 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:44:06.358421 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:45:26.890848 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d694cd-7950-4600-9d3a-db4bce2125a7 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:45:26.891483 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d694cd-7950-4600-9d3a-db4bce2125a7 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:45:26.891483 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d694cd-7950-4600-9d3a-db4bce2125a7 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:45:26.892723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 751/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 22:45:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:45:26.898746 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590e4b1-9147-45c0-8dcc-fb922b62f42e None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:45:26.899701 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590e4b1-9147-45c0-8dcc-fb922b62f42e None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:45:26.899701 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590e4b1-9147-45c0-8dcc-fb922b62f42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:45:26.980774 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-3590e4b1-9147-45c0-8dcc-fb922b62f42e None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:45:26.982018 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-3590e4b1-9147-45c0-8dcc-fb922b62f42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:45:27.091971 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 761/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 22:45:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:45:27.248776 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d694cd-7950-4600-9d3a-db4bce2125a7 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:45:27.251113 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-75d694cd-7950-4600-9d3a-db4bce2125a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:45:27.374010 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d694cd-7950-4600-9d3a-db4bce2125a7 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:45:27.379442 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ddbc75-90a7-41cb-9f8a-89c7f4de01a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:45:27.380044 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddbc75-90a7-41cb-9f8a-89c7f4de01a0 service nova] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:45:27.380421 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddbc75-90a7-41cb-9f8a-89c7f4de01a0 service nova] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:45:27.380698 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddbc75-90a7-41cb-9f8a-89c7f4de01a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:45:27.388693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ddbc75-90a7-41cb-9f8a-89c7f4de01a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:45:27.390312 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ddbc75-90a7-41cb-9f8a-89c7f4de01a0 service nova] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:45:27.399946 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 752/1513] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:45:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:45:28.247399 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e860907b-3ee8-4041-af60-6ab64bdb755c None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:45:28.247958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e860907b-3ee8-4041-af60-6ab64bdb755c None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:45:28.247958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e860907b-3ee8-4041-af60-6ab64bdb755c None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:45:28.249269 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 762/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 22:45:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:45:28.254216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02a6c4-3a4b-4a4d-a5cc-2a914776af53 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:45:28.254619 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02a6c4-3a4b-4a4d-a5cc-2a914776af53 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:45:28.254766 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02a6c4-3a4b-4a4d-a5cc-2a914776af53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:45:28.273947 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-ad02a6c4-3a4b-4a4d-a5cc-2a914776af53 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:45:28.275151 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-ad02a6c4-3a4b-4a4d-a5cc-2a914776af53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:45:28.283008 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 753/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 22:45:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:45:28.439222 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e860907b-3ee8-4041-af60-6ab64bdb755c None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:45:28.454012 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e860907b-3ee8-4041-af60-6ab64bdb755c None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:45:28.460480 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b39be-7a68-40b8-bb06-9409c1edf72c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:45:28.461085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b39be-7a68-40b8-bb06-9409c1edf72c service nova] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:45:28.461404 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b39be-7a68-40b8-bb06-9409c1edf72c service nova] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:45:28.461685 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b39be-7a68-40b8-bb06-9409c1edf72c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:45:28.470096 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b39be-7a68-40b8-bb06-9409c1edf72c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:45:28.471726 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b39be-7a68-40b8-bb06-9409c1edf72c service nova] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:45:28.482159 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 763/1516] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:45:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:45:29.971999 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16cadc-6605-404e-9773-ab4c2a81c820 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:45:29.972595 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16cadc-6605-404e-9773-ab4c2a81c820 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:45:29.972595 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16cadc-6605-404e-9773-ab4c2a81c820 None None] PATH_INFO: `/v3` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:45:29.973912 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 754/1517] 10.222.0.95 () {54 vars in 925 bytes} [Wed Aug 10 22:45:29 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:45:29.980310 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28615650-7ab0-490d-bf51-a901da269dfe None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:45:29.980776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28615650-7ab0-490d-bf51-a901da269dfe None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:45:29.980874 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28615650-7ab0-490d-bf51-a901da269dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:45:30.042799 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-28615650-7ab0-490d-bf51-a901da269dfe None None] MFA Rules not processed for user `9f46dee0f73e49eab54c11556e3b86bf`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:45:30.044035 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-28615650-7ab0-490d-bf51-a901da269dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:45:30.060636 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:45:30.115537 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 764/1518] 10.222.0.95 () {58 vars in 1015 bytes} [Wed Aug 10 22:45:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:45:34.073141 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0287857-f59b-4358-9b56-801c905dc382 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:45:34.073707 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0287857-f59b-4358-9b56-801c905dc382 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:45:34.073707 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0287857-f59b-4358-9b56-801c905dc382 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:45:34.074964 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 755/1519] 10.222.0.76 () {54 vars in 961 bytes} [Wed Aug 10 22:45:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:45:34.079903 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e7f82-9efd-4770-b52d-56e58fa66211 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:45:34.080287 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e7f82-9efd-4770-b52d-56e58fa66211 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:45:34.080452 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e7f82-9efd-4770-b52d-56e58fa66211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:45:34.099928 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-957e7f82-9efd-4770-b52d-56e58fa66211 None None] MFA Rules not processed for user `9f46dee0f73e49eab54c11556e3b86bf`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:45:34.101132 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-957e7f82-9efd-4770-b52d-56e58fa66211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:45:34.109680 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 765/1520] 10.222.0.76 () {58 vars in 1058 bytes} [Wed Aug 10 22:45:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:45:34.118070 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0287857-f59b-4358-9b56-801c905dc382 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:45:34.136366 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0287857-f59b-4358-9b56-801c905dc382 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:45:34.141523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b96a64-77f2-4c78-b3ae-cc7769f685f9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f46dee0f73e49eab54c11556e3b86bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:45:34.142123 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b96a64-77f2-4c78-b3ae-cc7769f685f9 service placement] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:45:34.142427 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b96a64-77f2-4c78-b3ae-cc7769f685f9 service placement] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:45:34.142692 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b96a64-77f2-4c78-b3ae-cc7769f685f9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:45:34.151150 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b96a64-77f2-4c78-b3ae-cc7769f685f9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:45:34.152780 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b96a64-77f2-4c78-b3ae-cc7769f685f9 service placement] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:45:34.162920 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 756/1521] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:45:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:45:39.713588 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ba457-1e15-4ae8-b0ec-5b85b177d271 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:45:39.714412 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ba457-1e15-4ae8-b0ec-5b85b177d271 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:45:39.714412 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ba457-1e15-4ae8-b0ec-5b85b177d271 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:45:39.715496 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 766/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 22:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:45:39.722669 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1da01b-96a6-4c1d-96c5-07b51791ef9d None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:45:39.723125 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1da01b-96a6-4c1d-96c5-07b51791ef9d None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:45:39.725793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1da01b-96a6-4c1d-96c5-07b51791ef9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:45:39.745783 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-fc1da01b-96a6-4c1d-96c5-07b51791ef9d None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:45:39.746977 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-fc1da01b-96a6-4c1d-96c5-07b51791ef9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:45:39.754732 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 757/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 22:45:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:45:39.907393 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8ba457-1e15-4ae8-b0ec-5b85b177d271 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:45:39.923052 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8ba457-1e15-4ae8-b0ec-5b85b177d271 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:45:39.929369 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646d0c7-63af-4fc3-8cc3-aea6060d4c84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:45:39.929976 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646d0c7-63af-4fc3-8cc3-aea6060d4c84 service nova] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:45:39.930258 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646d0c7-63af-4fc3-8cc3-aea6060d4c84 service nova] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:45:39.930533 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646d0c7-63af-4fc3-8cc3-aea6060d4c84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:45:39.938908 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c646d0c7-63af-4fc3-8cc3-aea6060d4c84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:45:39.940496 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c646d0c7-63af-4fc3-8cc3-aea6060d4c84 service nova] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:45:39.950962 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 767/1524] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:45:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:45:40.791074 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1293665-cb32-4227-b1cf-165571d57a48 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:45:40.791613 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1293665-cb32-4227-b1cf-165571d57a48 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:45:40.791613 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1293665-cb32-4227-b1cf-165571d57a48 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:45:40.792878 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 758/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 22:45:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:45:40.797624 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f0fa5-a0d5-4e15-be21-6eba4db298d7 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:45:40.798071 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f0fa5-a0d5-4e15-be21-6eba4db298d7 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:45:40.798182 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f0fa5-a0d5-4e15-be21-6eba4db298d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:45:40.818445 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-bc1f0fa5-a0d5-4e15-be21-6eba4db298d7 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:45:40.819717 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-bc1f0fa5-a0d5-4e15-be21-6eba4db298d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:45:40.827642 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 768/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 22:45:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:45:40.978776 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1293665-cb32-4227-b1cf-165571d57a48 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:45:40.993911 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1293665-cb32-4227-b1cf-165571d57a48 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:45:40.999256 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a162a41-7e91-4ff9-b44f-937bde994e26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:45:40.999848 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a162a41-7e91-4ff9-b44f-937bde994e26 service nova] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:45:41.000122 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a162a41-7e91-4ff9-b44f-937bde994e26 service nova] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:45:41.000390 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a162a41-7e91-4ff9-b44f-937bde994e26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:45:41.008629 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a162a41-7e91-4ff9-b44f-937bde994e26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:45:41.010252 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a162a41-7e91-4ff9-b44f-937bde994e26 service nova] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:45:41.019707 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 759/1527] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:45:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:45:44.954786 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d575dd0-2b43-42fd-bada-49fad76cc180 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:45:44.955315 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d575dd0-2b43-42fd-bada-49fad76cc180 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:45:44.955315 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d575dd0-2b43-42fd-bada-49fad76cc180 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:45:44.956669 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 769/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 22:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:45:44.962019 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9c567-b529-4a7c-a5e1-f39da2b5bfba None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:45:44.962455 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9c567-b529-4a7c-a5e1-f39da2b5bfba None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:45:44.962572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9c567-b529-4a7c-a5e1-f39da2b5bfba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:45:44.982650 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-0ca9c567-b529-4a7c-a5e1-f39da2b5bfba None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:45:44.983834 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-0ca9c567-b529-4a7c-a5e1-f39da2b5bfba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:45:44.992297 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 760/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 22:45:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:45:45.067613 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d575dd0-2b43-42fd-bada-49fad76cc180 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:45:45.070077 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-3d575dd0-2b43-42fd-bada-49fad76cc180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:45:45.148699 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d575dd0-2b43-42fd-bada-49fad76cc180 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:45:45.152325 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e1eee-4dcf-42ba-88ca-2fb6e8cac3e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:45:45.152932 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e1eee-4dcf-42ba-88ca-2fb6e8cac3e8 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:45:45.153213 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e1eee-4dcf-42ba-88ca-2fb6e8cac3e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:45:45.153516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e1eee-4dcf-42ba-88ca-2fb6e8cac3e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:45:45.162297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87e1eee-4dcf-42ba-88ca-2fb6e8cac3e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:45:45.163722 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87e1eee-4dcf-42ba-88ca-2fb6e8cac3e8 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:45:45.172971 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 770/1530] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:45:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:45:47.179476 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664dda3-b203-4065-a768-2d9c595b866c None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:45:47.180044 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664dda3-b203-4065-a768-2d9c595b866c None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:45:47.180044 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664dda3-b203-4065-a768-2d9c595b866c None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:45:47.181391 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 761/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 22:45:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:45:47.186746 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ba362-39a2-419f-97ae-c40256e42d6a None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:45:47.187189 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ba362-39a2-419f-97ae-c40256e42d6a None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:45:47.187287 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ba362-39a2-419f-97ae-c40256e42d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:45:47.207725 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-b99ba362-39a2-419f-97ae-c40256e42d6a None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:45:47.208936 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-b99ba362-39a2-419f-97ae-c40256e42d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:45:47.217354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 771/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 22:45:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:45:47.258595 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7664dda3-b203-4065-a768-2d9c595b866c None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:45:47.269190 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7664dda3-b203-4065-a768-2d9c595b866c None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:45:47.272730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012197b-d1b2-43e2-9d63-a2e77ca92058 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:45:47.273355 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012197b-d1b2-43e2-9d63-a2e77ca92058 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:45:47.273636 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012197b-d1b2-43e2-9d63-a2e77ca92058 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:45:47.273913 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012197b-d1b2-43e2-9d63-a2e77ca92058 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:45:47.282699 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9012197b-d1b2-43e2-9d63-a2e77ca92058 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:45:47.284133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9012197b-d1b2-43e2-9d63-a2e77ca92058 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:45:47.293615 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 762/1533] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:45:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:45:50.797209 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd0b8c-417f-4411-8dce-b58f5101c987 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:45:50.797768 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd0b8c-417f-4411-8dce-b58f5101c987 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:45:50.797768 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd0b8c-417f-4411-8dce-b58f5101c987 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:45:50.799061 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 772/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 22:45:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:45:50.804546 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342de5b7-fb93-4c38-b5df-b9777104af47 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:45:50.804965 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342de5b7-fb93-4c38-b5df-b9777104af47 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:45:50.805096 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342de5b7-fb93-4c38-b5df-b9777104af47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:45:50.825251 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-342de5b7-fb93-4c38-b5df-b9777104af47 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:45:50.826486 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-342de5b7-fb93-4c38-b5df-b9777104af47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:45:50.835349 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 763/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 22:45:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:45:51.140471 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd0b8c-417f-4411-8dce-b58f5101c987 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:45:51.142145 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-97dd0b8c-417f-4411-8dce-b58f5101c987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:45:51.209603 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:45:51.222915 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd0b8c-417f-4411-8dce-b58f5101c987 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:45:51.227813 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f2ee23-4720-455e-a3be-975ae773f524 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:45:51.228422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2ee23-4720-455e-a3be-975ae773f524 service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:45:51.228698 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2ee23-4720-455e-a3be-975ae773f524 service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:45:51.228976 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2ee23-4720-455e-a3be-975ae773f524 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:45:51.237918 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f2ee23-4720-455e-a3be-975ae773f524 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:45:51.239337 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f2ee23-4720-455e-a3be-975ae773f524 service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:45:51.248621 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 773/1536] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:45:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:45:53.304448 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471cef5-bcba-4e00-a29c-5105568011e0 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:45:53.304970 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471cef5-bcba-4e00-a29c-5105568011e0 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:45:53.304970 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471cef5-bcba-4e00-a29c-5105568011e0 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:45:53.306668 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 764/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 22:45:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:45:53.311738 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf97bf-5664-4943-8d36-42ba0e431088 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:45:53.312148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf97bf-5664-4943-8d36-42ba0e431088 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:45:53.312279 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf97bf-5664-4943-8d36-42ba0e431088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:45:53.331471 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-4edf97bf-5664-4943-8d36-42ba0e431088 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:45:53.332664 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-4edf97bf-5664-4943-8d36-42ba0e431088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:45:53.340966 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 774/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 22:45:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:45:53.646309 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e471cef5-bcba-4e00-a29c-5105568011e0 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:45:53.648064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-e471cef5-bcba-4e00-a29c-5105568011e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:45:53.661006 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e471cef5-bcba-4e00-a29c-5105568011e0 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:45:53.664779 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1fadf8-1e66-4433-8602-144ec6919329 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:45:53.665406 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1fadf8-1e66-4433-8602-144ec6919329 service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:45:53.666033 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1fadf8-1e66-4433-8602-144ec6919329 service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:45:53.666033 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1fadf8-1e66-4433-8602-144ec6919329 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:45:53.674795 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1fadf8-1e66-4433-8602-144ec6919329 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:45:53.676207 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1fadf8-1e66-4433-8602-144ec6919329 service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:45:53.685962 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 765/1539] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:45:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:46:03.326787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=11 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:46:03.681569 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=89 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:18.473415 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66efc7-a378-4e9f-b4c6-38c62e46c335 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:18.474639 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66efc7-a378-4e9f-b4c6-38c62e46c335 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:18.475121 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66efc7-a378-4e9f-b4c6-38c62e46c335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:18.504017 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-0c66efc7-a378-4e9f-b4c6-38c62e46c335 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:18.505264 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-0c66efc7-a378-4e9f-b4c6-38c62e46c335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:18.523615 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 775/1540] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:18.532767 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd79b0e-e118-4546-906f-f30744574cfd None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:18.533835 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd79b0e-e118-4546-906f-f30744574cfd None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:18.534401 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd79b0e-e118-4546-906f-f30744574cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:18.545292 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59884f-9713-44b8-8b92-d51c7ad6330b None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:18.545651 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59884f-9713-44b8-8b92-d51c7ad6330b None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:18.545922 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59884f-9713-44b8-8b92-d51c7ad6330b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:18.569153 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-7dd79b0e-e118-4546-906f-f30744574cfd None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:18.571279 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-7dd79b0e-e118-4546-906f-f30744574cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:18.582023 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-2b59884f-9713-44b8-8b92-d51c7ad6330b None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:18.583427 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-2b59884f-9713-44b8-8b92-d51c7ad6330b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:18.591782 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 766/1541] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:47:18.593544 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbaef6e-95eb-4921-ac01-154a6c577bed None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:18.594248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbaef6e-95eb-4921-ac01-154a6c577bed None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:18.594737 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbaef6e-95eb-4921-ac01-154a6c577bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:18.600080 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 776/1542] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:18.601001 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59884f-9713-44b8-8b92-d51c7ad6330b None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:18.619654 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59884f-9713-44b8-8b92-d51c7ad6330b None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:18.627943 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95e8949-2286-42f4-8189-d60dbd52c7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:18.628794 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e8949-2286-42f4-8189-d60dbd52c7a8 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:18.629080 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e8949-2286-42f4-8189-d60dbd52c7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:18.629448 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e8949-2286-42f4-8189-d60dbd52c7a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:18.629998 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-cdbaef6e-95eb-4921-ac01-154a6c577bed None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:18.630134 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95e8949-2286-42f4-8189-d60dbd52c7a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:47:18.630457 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95e8949-2286-42f4-8189-d60dbd52c7a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:18.631842 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-cdbaef6e-95eb-4921-ac01-154a6c577bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:18.632089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95e8949-2286-42f4-8189-d60dbd52c7a8 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:18.653650 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 767/1543] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:18.655334 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f311d-2be2-4824-a6c0-774d7e60342c None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:18.655740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f311d-2be2-4824-a6c0-774d7e60342c None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:18.656213 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f311d-2be2-4824-a6c0-774d7e60342c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:18.661686 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 777/1544] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:47:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:47:18.663228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a9418-a20b-47f2-9b40-d127bb208381 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:18.663581 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a9418-a20b-47f2-9b40-d127bb208381 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:18.663900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a9418-a20b-47f2-9b40-d127bb208381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:18.689882 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-176a9418-a20b-47f2-9b40-d127bb208381 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:18.691675 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-176a9418-a20b-47f2-9b40-d127bb208381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:18.696470 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-931f311d-2be2-4824-a6c0-774d7e60342c None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:18.698002 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-931f311d-2be2-4824-a6c0-774d7e60342c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:18.705312 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 778/1545] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:18.705312 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a9418-a20b-47f2-9b40-d127bb208381 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:18.720232 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 768/1546] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:18.721865 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc854e-cfc9-4eac-a414-2a9d97dcfbc2 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:18.722658 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc854e-cfc9-4eac-a414-2a9d97dcfbc2 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:18.723182 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc854e-cfc9-4eac-a414-2a9d97dcfbc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:18.724676 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a9418-a20b-47f2-9b40-d127bb208381 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:18.737680 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0a7a39-aa5b-43dc-ba72-f862648ae212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:18.738322 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a7a39-aa5b-43dc-ba72-f862648ae212 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:18.738794 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a7a39-aa5b-43dc-ba72-f862648ae212 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:18.739078 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a7a39-aa5b-43dc-ba72-f862648ae212 admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:18.742450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0a7a39-aa5b-43dc-ba72-f862648ae212 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:18.742450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: WARNING py.warnings [None req-5f0a7a39-aa5b-43dc-ba72-f862648ae212 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 22:47:18.742450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: warnings.warn(msg) Aug 10 22:47:18.742450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: Aug 10 22:47:18.744783 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0a7a39-aa5b-43dc-ba72-f862648ae212 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:18.774138 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-6cdc854e-cfc9-4eac-a414-2a9d97dcfbc2 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:18.777232 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-6cdc854e-cfc9-4eac-a414-2a9d97dcfbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:18.796346 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 769/1547] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:18.798045 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3ca38-f092-402f-9928-8f50e88fba2e None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:18.798616 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3ca38-f092-402f-9928-8f50e88fba2e None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:18.799229 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3ca38-f092-402f-9928-8f50e88fba2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:18.801050 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 779/1548] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:47:18 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:18.805544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0a7a39-aa5b-43dc-ba72-f862648ae212 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:18.846904 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-6ee3ca38-f092-402f-9928-8f50e88fba2e None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:18.850065 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-6ee3ca38-f092-402f-9928-8f50e88fba2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:18.905142 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0a7a39-aa5b-43dc-ba72-f862648ae212 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:18.910121 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82899b9-757d-4c60-b60d-cadcac737564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:18.910766 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82899b9-757d-4c60-b60d-cadcac737564 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:18.911058 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82899b9-757d-4c60-b60d-cadcac737564 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:18.911334 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82899b9-757d-4c60-b60d-cadcac737564 admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:18.912008 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82899b9-757d-4c60-b60d-cadcac737564 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:47:18.912310 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82899b9-757d-4c60-b60d-cadcac737564 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:18.914518 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82899b9-757d-4c60-b60d-cadcac737564 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:18.925930 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 770/1549] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:18.926214 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 780/1550] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:47:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:47:18.927185 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d31203-1706-442e-8bbe-8f311a5bd533 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:18.927317 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82899b9-757d-4c60-b60d-cadcac737564 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:18.927575 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d31203-1706-442e-8bbe-8f311a5bd533 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:18.927783 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d31203-1706-442e-8bbe-8f311a5bd533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:18.943035 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82899b9-757d-4c60-b60d-cadcac737564 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:18.947733 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f8b4b-d247-4a1d-b4ea-56ec579044ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:18.948341 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f8b4b-d247-4a1d-b4ea-56ec579044ee admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:18.948624 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f8b4b-d247-4a1d-b4ea-56ec579044ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:18.948900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f8b4b-d247-4a1d-b4ea-56ec579044ee admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:18.949595 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7f8b4b-d247-4a1d-b4ea-56ec579044ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:18.950100 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-45d31203-1706-442e-8bbe-8f311a5bd533 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:18.951310 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-45d31203-1706-442e-8bbe-8f311a5bd533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:18.951506 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7f8b4b-d247-4a1d-b4ea-56ec579044ee admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:18.959245 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 771/1551] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:18.960019 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d31203-1706-442e-8bbe-8f311a5bd533 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:18.970761 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d31203-1706-442e-8bbe-8f311a5bd533 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:18.977742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 781/1552] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:47:18 2022] POST /identity/v3/users => generated 293 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:18.977742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4e323-7557-47b0-a9d2-373bb158bb56 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:18.977742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4e323-7557-47b0-a9d2-373bb158bb56 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:18.977742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4e323-7557-47b0-a9d2-373bb158bb56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:18.980146 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736f270-6ce4-4437-9a6e-04172a63f8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:18.980778 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736f270-6ce4-4437-9a6e-04172a63f8ca admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:18.981061 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736f270-6ce4-4437-9a6e-04172a63f8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:18.981361 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736f270-6ce4-4437-9a6e-04172a63f8ca admin admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:18.982016 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1736f270-6ce4-4437-9a6e-04172a63f8ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:47:18.982317 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1736f270-6ce4-4437-9a6e-04172a63f8ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:18.983337 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: WARNING py.warnings [None req-1736f270-6ce4-4437-9a6e-04172a63f8ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 22:47:18.983337 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: warnings.warn(msg) Aug 10 22:47:18.983337 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: Aug 10 22:47:18.984226 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1736f270-6ce4-4437-9a6e-04172a63f8ca admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:18.996819 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 772/1553] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:47:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:47:18.998029 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38aa9b5-bf11-4afb-a678-8ea621951483 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:18.998315 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38aa9b5-bf11-4afb-a678-8ea621951483 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:18.998446 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-88f4e323-7557-47b0-a9d2-373bb158bb56 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:18.998617 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38aa9b5-bf11-4afb-a678-8ea621951483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:18.999657 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-88f4e323-7557-47b0-a9d2-373bb158bb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:19.008126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 782/1554] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:19.009419 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f98d4-5bd9-4e77-90be-cb3be9a2cb09 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:19.009701 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f98d4-5bd9-4e77-90be-cb3be9a2cb09 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:19.009971 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f98d4-5bd9-4e77-90be-cb3be9a2cb09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:19.018528 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-e38aa9b5-bf11-4afb-a678-8ea621951483 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:19.019735 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-e38aa9b5-bf11-4afb-a678-8ea621951483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:19.027918 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 773/1555] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:19.028677 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38aa9b5-bf11-4afb-a678-8ea621951483 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:19.029133 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-b70f98d4-5bd9-4e77-90be-cb3be9a2cb09 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:19.030594 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-b70f98d4-5bd9-4e77-90be-cb3be9a2cb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:19.038364 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 783/1556] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:19.039769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38aa9b5-bf11-4afb-a678-8ea621951483 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:19.040322 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f98d4-5bd9-4e77-90be-cb3be9a2cb09 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:19.044791 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458e7af-5e9f-451d-8dbb-bf9c3b2010a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:19.045657 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458e7af-5e9f-451d-8dbb-bf9c3b2010a5 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:19.046111 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458e7af-5e9f-451d-8dbb-bf9c3b2010a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:19.046640 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458e7af-5e9f-451d-8dbb-bf9c3b2010a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:19.047227 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8458e7af-5e9f-451d-8dbb-bf9c3b2010a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:19.049475 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8458e7af-5e9f-451d-8dbb-bf9c3b2010a5 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:19.051040 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f98d4-5bd9-4e77-90be-cb3be9a2cb09 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:19.058061 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cfb723-c59d-4e6a-aa67-126bf5699cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:19.060293 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cfb723-c59d-4e6a-aa67-126bf5699cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:19.060769 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cfb723-c59d-4e6a-aa67-126bf5699cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:19.061188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cfb723-c59d-4e6a-aa67-126bf5699cc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:19.062313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cfb723-c59d-4e6a-aa67-126bf5699cc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:47:19.062824 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cfb723-c59d-4e6a-aa67-126bf5699cc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:19.065507 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cfb723-c59d-4e6a-aa67-126bf5699cc3 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:19.082395 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 784/1557] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:47:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:47:19.083879 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e513c57-beb3-4827-99e2-055ae8ff3bd3 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:19.084798 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e513c57-beb3-4827-99e2-055ae8ff3bd3 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:19.085086 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e513c57-beb3-4827-99e2-055ae8ff3bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:19.094883 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 774/1558] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:47:19 2022] POST /identity/v3/projects => generated 363 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:19.095971 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458e7af-5e9f-451d-8dbb-bf9c3b2010a5 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:19.108457 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-4e513c57-beb3-4827-99e2-055ae8ff3bd3 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:19.109696 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-4e513c57-beb3-4827-99e2-055ae8ff3bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:19.152044 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:19.159696 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458e7af-5e9f-451d-8dbb-bf9c3b2010a5 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:19.165961 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861b09f-31cc-4853-92b8-670bfcbb8bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:19.167773 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861b09f-31cc-4853-92b8-670bfcbb8bbc admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:19.168057 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861b09f-31cc-4853-92b8-670bfcbb8bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:19.168335 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861b09f-31cc-4853-92b8-670bfcbb8bbc admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:19.169162 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1861b09f-31cc-4853-92b8-670bfcbb8bbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:19.172032 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1861b09f-31cc-4853-92b8-670bfcbb8bbc admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:19.175429 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 785/1559] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:19.176349 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e513c57-beb3-4827-99e2-055ae8ff3bd3 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:19.184291 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 775/1560] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:19.185549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8d677-950f-4894-af23-eddf0452b219 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:19.185829 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8d677-950f-4894-af23-eddf0452b219 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:19.186099 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8d677-950f-4894-af23-eddf0452b219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:19.187248 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e513c57-beb3-4827-99e2-055ae8ff3bd3 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:19.192495 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9564d-10c4-44ff-96ed-bb585292f562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:19.193171 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9564d-10c4-44ff-96ed-bb585292f562 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:19.193408 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9564d-10c4-44ff-96ed-bb585292f562 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:19.193668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9564d-10c4-44ff-96ed-bb585292f562 admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:19.194295 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9564d-10c4-44ff-96ed-bb585292f562 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:19.196432 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9564d-10c4-44ff-96ed-bb585292f562 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:19.207581 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-47e8d677-950f-4894-af23-eddf0452b219 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:19.208775 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-47e8d677-950f-4894-af23-eddf0452b219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:19.217048 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 776/1561] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:19.218319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3654ca-2461-4d8a-9703-d05a55d1172f None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:19.218589 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3654ca-2461-4d8a-9703-d05a55d1172f None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:19.218856 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3654ca-2461-4d8a-9703-d05a55d1172f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:19.228865 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 786/1562] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:47:19 2022] POST /identity/v3/projects => generated 405 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:19.230034 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb452c2-2276-4a61-814b-727e8630f6b3 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:19.230334 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb452c2-2276-4a61-814b-727e8630f6b3 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:19.230614 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb452c2-2276-4a61-814b-727e8630f6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:19.242872 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-3a3654ca-2461-4d8a-9703-d05a55d1172f None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:19.244088 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-3a3654ca-2461-4d8a-9703-d05a55d1172f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:19.250988 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-7bb452c2-2276-4a61-814b-727e8630f6b3 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:19.252185 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-7bb452c2-2276-4a61-814b-727e8630f6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:19.253670 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:19.312821 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 777/1563] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:19.313701 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3654ca-2461-4d8a-9703-d05a55d1172f None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:19.317523 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 787/1564] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:19.317523 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb452c2-2276-4a61-814b-727e8630f6b3 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:19.327075 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3654ca-2461-4d8a-9703-d05a55d1172f None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:19.327319 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb452c2-2276-4a61-814b-727e8630f6b3 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:19.331613 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2907f1b0-17c5-45ae-9cdf-c02058be2c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:19.332212 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907f1b0-17c5-45ae-9cdf-c02058be2c6c admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:19.332386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490cbbe-429c-4e19-b7d5-c784f4dae469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:19.332510 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907f1b0-17c5-45ae-9cdf-c02058be2c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:19.332737 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907f1b0-17c5-45ae-9cdf-c02058be2c6c admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:19.333190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490cbbe-429c-4e19-b7d5-c784f4dae469 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:19.333379 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2907f1b0-17c5-45ae-9cdf-c02058be2c6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:19.333651 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490cbbe-429c-4e19-b7d5-c784f4dae469 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:19.334055 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490cbbe-429c-4e19-b7d5-c784f4dae469 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:19.334689 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5490cbbe-429c-4e19-b7d5-c784f4dae469 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:19.335360 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2907f1b0-17c5-45ae-9cdf-c02058be2c6c admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:19.336716 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5490cbbe-429c-4e19-b7d5-c784f4dae469 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:19.361571 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 788/1565] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:47:19 2022] POST /identity/v3/users => generated 282 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:19.362642 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490cbbe-429c-4e19-b7d5-c784f4dae469 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:19.367951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 778/1566] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:47:19 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:19.368750 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2907f1b0-17c5-45ae-9cdf-c02058be2c6c admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:19.373152 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490cbbe-429c-4e19-b7d5-c784f4dae469 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:19.433451 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374b71c-5459-444d-a59e-f7ba01d43efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:19.433618 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374b71c-5459-444d-a59e-f7ba01d43efa admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:19.433829 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374b71c-5459-444d-a59e-f7ba01d43efa admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:19.434075 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374b71c-5459-444d-a59e-f7ba01d43efa admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:19.434676 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2907f1b0-17c5-45ae-9cdf-c02058be2c6c admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:19.434786 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a374b71c-5459-444d-a59e-f7ba01d43efa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:47:19.435003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a374b71c-5459-444d-a59e-f7ba01d43efa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:19.436609 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a374b71c-5459-444d-a59e-f7ba01d43efa admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:19.439632 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4bd26-69fd-49a8-9a3d-a15cd45a2343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:19.440243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4bd26-69fd-49a8-9a3d-a15cd45a2343 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:19.440520 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4bd26-69fd-49a8-9a3d-a15cd45a2343 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:19.440790 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4bd26-69fd-49a8-9a3d-a15cd45a2343 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:19.441474 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4bd26-69fd-49a8-9a3d-a15cd45a2343 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:19.443085 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4bd26-69fd-49a8-9a3d-a15cd45a2343 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:19.448325 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 789/1567] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:47:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:47:19.449088 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374b71c-5459-444d-a59e-f7ba01d43efa admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:19.458585 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374b71c-5459-444d-a59e-f7ba01d43efa admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:19.461144 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 779/1568] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:19.461898 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4bd26-69fd-49a8-9a3d-a15cd45a2343 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:19.463767 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954ef8c-95e6-4015-bf26-721776706033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:19.464370 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954ef8c-95e6-4015-bf26-721776706033 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:19.464649 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954ef8c-95e6-4015-bf26-721776706033 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:19.464925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954ef8c-95e6-4015-bf26-721776706033 admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:19.465598 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b954ef8c-95e6-4015-bf26-721776706033 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:47:19.465902 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b954ef8c-95e6-4015-bf26-721776706033 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:19.467501 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b954ef8c-95e6-4015-bf26-721776706033 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:19.472121 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4bd26-69fd-49a8-9a3d-a15cd45a2343 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:19.477736 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef15283-3f35-4ced-8d95-c6a519941f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:19.478400 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef15283-3f35-4ced-8d95-c6a519941f34 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:19.478680 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef15283-3f35-4ced-8d95-c6a519941f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:19.478948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef15283-3f35-4ced-8d95-c6a519941f34 admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:19.479676 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef15283-3f35-4ced-8d95-c6a519941f34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:19.480657 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 790/1569] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:47:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:47:19.481134 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: WARNING py.warnings [None req-9ef15283-3f35-4ced-8d95-c6a519941f34 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 22:47:19.481134 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: warnings.warn(msg) Aug 10 22:47:19.481134 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: Aug 10 22:47:19.481577 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954ef8c-95e6-4015-bf26-721776706033 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:19.481918 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef15283-3f35-4ced-8d95-c6a519941f34 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:19.492852 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954ef8c-95e6-4015-bf26-721776706033 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:19.497714 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c26cb8-cb5d-4b60-b368-f3bc082aaa60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:19.498309 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c26cb8-cb5d-4b60-b368-f3bc082aaa60 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:19.498585 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c26cb8-cb5d-4b60-b368-f3bc082aaa60 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:19.498856 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c26cb8-cb5d-4b60-b368-f3bc082aaa60 admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:19.499489 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c26cb8-cb5d-4b60-b368-f3bc082aaa60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:47:19.499799 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c26cb8-cb5d-4b60-b368-f3bc082aaa60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:19.501411 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c26cb8-cb5d-4b60-b368-f3bc082aaa60 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:19.504608 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 780/1570] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:47:19 2022] POST /identity/v3/users => generated 303 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:19.505395 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef15283-3f35-4ced-8d95-c6a519941f34 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:19.513788 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 791/1571] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:47:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:47:19.514718 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c26cb8-cb5d-4b60-b368-f3bc082aaa60 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:19.516587 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef15283-3f35-4ced-8d95-c6a519941f34 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:19.522034 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fdc880-7207-47f0-a03d-3b5eddecf378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:19.522685 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdc880-7207-47f0-a03d-3b5eddecf378 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:19.523806 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdc880-7207-47f0-a03d-3b5eddecf378 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:19.524079 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdc880-7207-47f0-a03d-3b5eddecf378 admin admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:19.524780 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fdc880-7207-47f0-a03d-3b5eddecf378 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:47:19.525106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fdc880-7207-47f0-a03d-3b5eddecf378 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:19.525713 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c26cb8-cb5d-4b60-b368-f3bc082aaa60 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:19.527392 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fdc880-7207-47f0-a03d-3b5eddecf378 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:19.530963 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c49a135-d908-4234-bc75-c91a97f0831e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:19.531564 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49a135-d908-4234-bc75-c91a97f0831e admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:19.531870 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49a135-d908-4234-bc75-c91a97f0831e admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:19.532154 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49a135-d908-4234-bc75-c91a97f0831e admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:19.532794 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c49a135-d908-4234-bc75-c91a97f0831e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:19.534427 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c49a135-d908-4234-bc75-c91a97f0831e admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:19.541137 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 781/1572] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:47:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:47:19.542087 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fdc880-7207-47f0-a03d-3b5eddecf378 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:19.547014 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 792/1573] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:19.550648 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c49a135-d908-4234-bc75-c91a97f0831e admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:19.552191 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fdc880-7207-47f0-a03d-3b5eddecf378 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:19.557067 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417821c-376f-4f34-8764-188feed39ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:19.557692 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417821c-376f-4f34-8764-188feed39ddb admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:19.557968 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417821c-376f-4f34-8764-188feed39ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:19.558117 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:19.558226 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417821c-376f-4f34-8764-188feed39ddb admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:19.558830 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3417821c-376f-4f34-8764-188feed39ddb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:19.560741 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3417821c-376f-4f34-8764-188feed39ddb admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:19.561896 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c49a135-d908-4234-bc75-c91a97f0831e admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:19.567418 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa4d54-7e85-488c-8758-e811fe4f14f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:19.568008 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa4d54-7e85-488c-8758-e811fe4f14f0 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:19.568286 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa4d54-7e85-488c-8758-e811fe4f14f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:19.568561 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa4d54-7e85-488c-8758-e811fe4f14f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:19.569176 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa4d54-7e85-488c-8758-e811fe4f14f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:19.571380 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa4d54-7e85-488c-8758-e811fe4f14f0 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:19.584715 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 782/1574] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:47:19 2022] POST /identity/v3/users => generated 286 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:19.584715 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417821c-376f-4f34-8764-188feed39ddb admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:19.593888 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417821c-376f-4f34-8764-188feed39ddb admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:19.598506 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 793/1575] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:47:19 2022] POST /identity/v3/projects => generated 389 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:19.598769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32f155-8506-4442-8a3e-7f39a68c6264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:19.599270 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32f155-8506-4442-8a3e-7f39a68c6264 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:19.599553 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32f155-8506-4442-8a3e-7f39a68c6264 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:19.599825 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32f155-8506-4442-8a3e-7f39a68c6264 admin admin] PATH_INFO: `/v3/projects/dfa4e852819140a8ad723f54c756f607/users/80a65ec30ace42ceb45e1a2d9d4c0b12/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:19.602585 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa4d54-7e85-488c-8758-e811fe4f14f0 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:19.615358 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd32f155-8506-4442-8a3e-7f39a68c6264 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfa4e852819140a8ad723f54c756f607, user_id=80a65ec30ace42ceb45e1a2d9d4c0b12, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:19.618625 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd32f155-8506-4442-8a3e-7f39a68c6264 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:19.631574 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 783/1576] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:47:19 2022] PUT /identity/v3/projects/dfa4e852819140a8ad723f54c756f607/users/80a65ec30ace42ceb45e1a2d9d4c0b12/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:19.632486 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32f155-8506-4442-8a3e-7f39a68c6264 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:19.660594 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:19.667840 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa4d54-7e85-488c-8758-e811fe4f14f0 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:19.707882 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32f155-8506-4442-8a3e-7f39a68c6264 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:19.714096 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f995481-508a-475a-b1ef-a722e65e1068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:19.714817 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f995481-508a-475a-b1ef-a722e65e1068 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:19.715091 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f995481-508a-475a-b1ef-a722e65e1068 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:19.715357 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f995481-508a-475a-b1ef-a722e65e1068 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:19.716138 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f995481-508a-475a-b1ef-a722e65e1068 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:19.717784 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f995481-508a-475a-b1ef-a722e65e1068 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:19.728607 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 784/1577] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:19.729199 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f995481-508a-475a-b1ef-a722e65e1068 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:19.735298 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1903e9e-cda6-4be1-8e35-6436baf0b884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:19.735921 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1903e9e-cda6-4be1-8e35-6436baf0b884 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:19.736201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1903e9e-cda6-4be1-8e35-6436baf0b884 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:19.736476 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1903e9e-cda6-4be1-8e35-6436baf0b884 admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:19.737104 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1903e9e-cda6-4be1-8e35-6436baf0b884 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:19.739092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1903e9e-cda6-4be1-8e35-6436baf0b884 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:19.739317 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f995481-508a-475a-b1ef-a722e65e1068 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:19.744921 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223330ac-3309-4e80-8692-1d1357afd817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:19.745565 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223330ac-3309-4e80-8692-1d1357afd817 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:19.745874 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223330ac-3309-4e80-8692-1d1357afd817 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:19.746271 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223330ac-3309-4e80-8692-1d1357afd817 admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:19.746929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223330ac-3309-4e80-8692-1d1357afd817 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:19.748857 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223330ac-3309-4e80-8692-1d1357afd817 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:19.768505 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 794/1578] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:47:19 2022] POST /identity/v3/projects => generated 383 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:19.769167 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1903e9e-cda6-4be1-8e35-6436baf0b884 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:19.781050 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 785/1579] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:47:19 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:19.785440 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223330ac-3309-4e80-8692-1d1357afd817 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:19.832996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1903e9e-cda6-4be1-8e35-6436baf0b884 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:19.851715 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:19.856515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223330ac-3309-4e80-8692-1d1357afd817 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:19.861724 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64346c-a086-4651-ae95-dd6c34fbb020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:19.862340 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64346c-a086-4651-ae95-dd6c34fbb020 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:19.862630 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64346c-a086-4651-ae95-dd6c34fbb020 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:19.862905 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64346c-a086-4651-ae95-dd6c34fbb020 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:19.863546 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e64346c-a086-4651-ae95-dd6c34fbb020 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:19.865172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e64346c-a086-4651-ae95-dd6c34fbb020 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:19.876236 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 786/1580] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:19.877879 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64346c-a086-4651-ae95-dd6c34fbb020 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:19.887900 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64346c-a086-4651-ae95-dd6c34fbb020 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:19.894207 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d7b3d-7d4a-47d1-acaa-95aaf000b1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:19.894891 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d7b3d-7d4a-47d1-acaa-95aaf000b1f5 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:19.895080 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d7b3d-7d4a-47d1-acaa-95aaf000b1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:19.895371 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d7b3d-7d4a-47d1-acaa-95aaf000b1f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:19.896015 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d7b3d-7d4a-47d1-acaa-95aaf000b1f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:19.896347 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c3d0a-0350-4825-8160-03ce04fe804d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:19.897059 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c3d0a-0350-4825-8160-03ce04fe804d admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:19.897392 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c3d0a-0350-4825-8160-03ce04fe804d admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:19.897737 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c3d0a-0350-4825-8160-03ce04fe804d admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:19.897944 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d7b3d-7d4a-47d1-acaa-95aaf000b1f5 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:19.898627 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841c3d0a-0350-4825-8160-03ce04fe804d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:19.901732 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841c3d0a-0350-4825-8160-03ce04fe804d admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:19.908950 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 787/1581] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:19.909797 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d7b3d-7d4a-47d1-acaa-95aaf000b1f5 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:19.921829 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d7b3d-7d4a-47d1-acaa-95aaf000b1f5 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:19.927130 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7751915-efbd-4a6e-9f27-9b2242a73dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:19.927732 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7751915-efbd-4a6e-9f27-9b2242a73dd5 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:19.928012 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7751915-efbd-4a6e-9f27-9b2242a73dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:19.928286 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7751915-efbd-4a6e-9f27-9b2242a73dd5 admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:19.929732 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7751915-efbd-4a6e-9f27-9b2242a73dd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:19.931792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7751915-efbd-4a6e-9f27-9b2242a73dd5 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:19.935392 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 795/1582] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:47:19 2022] POST /identity/v3/projects => generated 375 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:19.935392 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f5d78-4365-49ad-8edc-3f64f92205d0 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:19.935392 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f5d78-4365-49ad-8edc-3f64f92205d0 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:19.935392 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f5d78-4365-49ad-8edc-3f64f92205d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:19.955872 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 788/1583] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:47:19 2022] POST /identity/v3/users => generated 295 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:19.957072 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7751915-efbd-4a6e-9f27-9b2242a73dd5 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:19.963429 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-a66f5d78-4365-49ad-8edc-3f64f92205d0 None None] MFA Rules not processed for user `80a65ec30ace42ceb45e1a2d9d4c0b12`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:19.964579 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-a66f5d78-4365-49ad-8edc-3f64f92205d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:19.968097 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7751915-efbd-4a6e-9f27-9b2242a73dd5 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:20.025789 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb08d75-3b93-45a3-a518-adaa9b0f1ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:20.026354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb08d75-3b93-45a3-a518-adaa9b0f1ccb admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:20.026725 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb08d75-3b93-45a3-a518-adaa9b0f1ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:20.026977 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb08d75-3b93-45a3-a518-adaa9b0f1ccb admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:20.027701 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb08d75-3b93-45a3-a518-adaa9b0f1ccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:20.029246 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb08d75-3b93-45a3-a518-adaa9b0f1ccb admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:20.034844 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 796/1584] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:20.034844 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f5d78-4365-49ad-8edc-3f64f92205d0 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:20.037728 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:20.044586 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 789/1585] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:20.046260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb08d75-3b93-45a3-a518-adaa9b0f1ccb admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:20.046372 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f5d78-4365-49ad-8edc-3f64f92205d0 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:20.052323 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f853e-dcec-4bb4-950f-5f540aa2d3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:20.052951 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f853e-dcec-4bb4-950f-5f540aa2d3a5 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:20.053233 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f853e-dcec-4bb4-950f-5f540aa2d3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:20.053805 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f853e-dcec-4bb4-950f-5f540aa2d3a5 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:20.056022 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532f853e-dcec-4bb4-950f-5f540aa2d3a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:20.058578 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532f853e-dcec-4bb4-950f-5f540aa2d3a5 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:20.058906 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb08d75-3b93-45a3-a518-adaa9b0f1ccb admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:20.064026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96f851-6ce5-4e70-8950-9e8dc52bf274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:20.065013 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96f851-6ce5-4e70-8950-9e8dc52bf274 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:20.065453 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96f851-6ce5-4e70-8950-9e8dc52bf274 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:20.065734 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96f851-6ce5-4e70-8950-9e8dc52bf274 admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:20.066358 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba96f851-6ce5-4e70-8950-9e8dc52bf274 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:20.068266 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba96f851-6ce5-4e70-8950-9e8dc52bf274 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:20.093620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 790/1586] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:47:20 2022] POST /identity/v3/users => generated 290 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:20.097419 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96f851-6ce5-4e70-8950-9e8dc52bf274 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:20.100418 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 797/1587] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:47:20 2022] POST /identity/v3/users => generated 292 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:20.102786 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f853e-dcec-4bb4-950f-5f540aa2d3a5 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:20.108669 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96f851-6ce5-4e70-8950-9e8dc52bf274 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:20.113474 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa285b4-f6c8-457a-a4b2-ad813b659ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:20.114128 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa285b4-f6c8-457a-a4b2-ad813b659ebb admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:20.114408 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa285b4-f6c8-457a-a4b2-ad813b659ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:20.114674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa285b4-f6c8-457a-a4b2-ad813b659ebb admin admin] PATH_INFO: `/v3/projects/09cce397513c41ceb2280e51b44b1a4d/users/ffd10984bde54bb586ecc86386ae3b19/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:20.116139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f853e-dcec-4bb4-950f-5f540aa2d3a5 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:20.121875 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92caf08-4140-4852-b4ca-668dba087abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:20.122498 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92caf08-4140-4852-b4ca-668dba087abf admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:20.122785 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92caf08-4140-4852-b4ca-668dba087abf admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:20.123061 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92caf08-4140-4852-b4ca-668dba087abf admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:20.123719 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92caf08-4140-4852-b4ca-668dba087abf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:20.125804 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92caf08-4140-4852-b4ca-668dba087abf admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:20.131353 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa285b4-f6c8-457a-a4b2-ad813b659ebb admin admin] RBAC: Authorizing `identity:create_grant(project_id=09cce397513c41ceb2280e51b44b1a4d, user_id=ffd10984bde54bb586ecc86386ae3b19, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:20.134693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa285b4-f6c8-457a-a4b2-ad813b659ebb admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:20.137523 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 798/1588] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:20.141350 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92caf08-4140-4852-b4ca-668dba087abf admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:20.151168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 791/1589] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:47:20 2022] PUT /identity/v3/projects/09cce397513c41ceb2280e51b44b1a4d/users/ffd10984bde54bb586ecc86386ae3b19/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:20.151912 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92caf08-4140-4852-b4ca-668dba087abf admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:20.154129 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa285b4-f6c8-457a-a4b2-ad813b659ebb admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:20.210309 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29199976-97b0-4ccb-9d4f-8c0c7d3e4022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:20.210928 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29199976-97b0-4ccb-9d4f-8c0c7d3e4022 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:20.211202 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29199976-97b0-4ccb-9d4f-8c0c7d3e4022 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:20.211471 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29199976-97b0-4ccb-9d4f-8c0c7d3e4022 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:20.212111 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29199976-97b0-4ccb-9d4f-8c0c7d3e4022 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:20.214129 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29199976-97b0-4ccb-9d4f-8c0c7d3e4022 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:20.217969 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa285b4-f6c8-457a-a4b2-ad813b659ebb admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:20.222891 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c08f16a-effc-4f1b-8514-e775efe42cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:20.223501 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08f16a-effc-4f1b-8514-e775efe42cba admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:20.223775 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08f16a-effc-4f1b-8514-e775efe42cba admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:20.224053 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08f16a-effc-4f1b-8514-e775efe42cba admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:20.224758 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c08f16a-effc-4f1b-8514-e775efe42cba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:20.226471 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c08f16a-effc-4f1b-8514-e775efe42cba admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:20.230480 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:20.236935 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 799/1590] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:47:20 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:20.237405 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 792/1591] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:20 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:20.238667 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14712e00-3530-4cbf-b224-ace9fe564351 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:20.239070 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14712e00-3530-4cbf-b224-ace9fe564351 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:20.239454 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14712e00-3530-4cbf-b224-ace9fe564351 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:20.240711 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29199976-97b0-4ccb-9d4f-8c0c7d3e4022 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:20.240995 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 793/1592] 10.222.0.76 () {54 vars in 962 bytes} [Wed Aug 10 22:47:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:47:20.246143 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14712e00-3530-4cbf-b224-ace9fe564351 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:20.252515 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29199976-97b0-4ccb-9d4f-8c0c7d3e4022 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:20.257257 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31421d78-a50c-4efb-8714-dcdfff84c9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:20.258135 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31421d78-a50c-4efb-8714-dcdfff84c9d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:20.258424 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31421d78-a50c-4efb-8714-dcdfff84c9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:20.258704 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31421d78-a50c-4efb-8714-dcdfff84c9d7 admin admin] PATH_INFO: `/v3/projects/5335171be4fe4b9189674285700e696b/users/20734b109838409192d9769a5a5de60b/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:20.259869 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14712e00-3530-4cbf-b224-ace9fe564351 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:20.264385 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce61480-9c71-4467-b6a3-0dd55b7d2a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:20.264982 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce61480-9c71-4467-b6a3-0dd55b7d2a33 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:20.265259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce61480-9c71-4467-b6a3-0dd55b7d2a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:20.265559 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce61480-9c71-4467-b6a3-0dd55b7d2a33 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:20.266191 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce61480-9c71-4467-b6a3-0dd55b7d2a33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:20.267789 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce61480-9c71-4467-b6a3-0dd55b7d2a33 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:20.274364 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31421d78-a50c-4efb-8714-dcdfff84c9d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5335171be4fe4b9189674285700e696b, user_id=20734b109838409192d9769a5a5de60b, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:20.275423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: WARNING py.warnings [None req-31421d78-a50c-4efb-8714-dcdfff84c9d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 22:47:20.275423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: warnings.warn(msg) Aug 10 22:47:20.275423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: Aug 10 22:47:20.277377 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31421d78-a50c-4efb-8714-dcdfff84c9d7 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:20.278960 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 794/1593] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:20.279760 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce61480-9c71-4467-b6a3-0dd55b7d2a33 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:20.290536 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce61480-9c71-4467-b6a3-0dd55b7d2a33 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:20.296795 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 800/1594] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:47:20 2022] PUT /identity/v3/projects/5335171be4fe4b9189674285700e696b/users/20734b109838409192d9769a5a5de60b/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:20.297808 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31421d78-a50c-4efb-8714-dcdfff84c9d7 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:20.350698 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5aa3f0-d39b-4d60-ab30-7191492edd03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:20.351511 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5aa3f0-d39b-4d60-ab30-7191492edd03 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:20.351776 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5aa3f0-d39b-4d60-ab30-7191492edd03 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:20.352042 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5aa3f0-d39b-4d60-ab30-7191492edd03 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:20.352666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5aa3f0-d39b-4d60-ab30-7191492edd03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:20.354294 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5aa3f0-d39b-4d60-ab30-7191492edd03 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:20.361584 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31421d78-a50c-4efb-8714-dcdfff84c9d7 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:20.363890 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 795/1595] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:20 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:20.365198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cba735-d6c0-4b8b-939f-0b6cf9d79b38 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:20.365505 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cba735-d6c0-4b8b-939f-0b6cf9d79b38 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:20.365781 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cba735-d6c0-4b8b-939f-0b6cf9d79b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:20.370348 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4829f-6b3c-4b6b-bee0-ed65f52fa5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:20.371115 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4829f-6b3c-4b6b-bee0-ed65f52fa5ce admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:20.371475 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4829f-6b3c-4b6b-bee0-ed65f52fa5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:20.371844 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4829f-6b3c-4b6b-bee0-ed65f52fa5ce admin admin] PATH_INFO: `/v3/projects/2daad69c6ae0460db7ac5c9df6efd7a0/users/6e9467aadb8e4b1794a8fef711ea1f88/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:20.387088 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f4829f-6b3c-4b6b-bee0-ed65f52fa5ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=2daad69c6ae0460db7ac5c9df6efd7a0, user_id=6e9467aadb8e4b1794a8fef711ea1f88, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:20.389818 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f4829f-6b3c-4b6b-bee0-ed65f52fa5ce admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:20.393228 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-f2cba735-d6c0-4b8b-939f-0b6cf9d79b38 None None] MFA Rules not processed for user `ffd10984bde54bb586ecc86386ae3b19`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:20.394464 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-f2cba735-d6c0-4b8b-939f-0b6cf9d79b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:20.403822 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 801/1596] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:47:20 2022] PUT /identity/v3/projects/2daad69c6ae0460db7ac5c9df6efd7a0/users/6e9467aadb8e4b1794a8fef711ea1f88/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:20.404640 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4829f-6b3c-4b6b-bee0-ed65f52fa5ce admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:20.424017 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:20.466526 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 796/1597] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:20.467360 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cba735-d6c0-4b8b-939f-0b6cf9d79b38 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:20.478228 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cba735-d6c0-4b8b-939f-0b6cf9d79b38 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:20.482389 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4829f-6b3c-4b6b-bee0-ed65f52fa5ce admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:20.482859 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a54ca7-fa07-41fd-9909-d11ae8a9ff89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:20.483488 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a54ca7-fa07-41fd-9909-d11ae8a9ff89 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:20.483763 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a54ca7-fa07-41fd-9909-d11ae8a9ff89 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:20.484030 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a54ca7-fa07-41fd-9909-d11ae8a9ff89 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:20.484667 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a54ca7-fa07-41fd-9909-d11ae8a9ff89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:20.486283 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a54ca7-fa07-41fd-9909-d11ae8a9ff89 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:20.487173 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b81b3-56a1-440a-ad65-0da7d375e875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:20.487787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b81b3-56a1-440a-ad65-0da7d375e875 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:20.488069 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b81b3-56a1-440a-ad65-0da7d375e875 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:20.488353 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b81b3-56a1-440a-ad65-0da7d375e875 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:20.489024 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b81b3-56a1-440a-ad65-0da7d375e875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:20.490788 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b81b3-56a1-440a-ad65-0da7d375e875 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:20.497328 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 797/1598] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:20.498755 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ebf140-1bf9-439c-81f1-5acbbe0fb90d None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:20.499040 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ebf140-1bf9-439c-81f1-5acbbe0fb90d None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:20.499309 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ebf140-1bf9-439c-81f1-5acbbe0fb90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:20.503548 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 802/1599] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:20 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:20.506211 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b81b3-56a1-440a-ad65-0da7d375e875 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:20.516267 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b81b3-56a1-440a-ad65-0da7d375e875 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:20.521001 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81ed1a6-93d3-46c8-a7ec-1d96aa2772c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:20.521628 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ed1a6-93d3-46c8-a7ec-1d96aa2772c2 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:20.521907 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ed1a6-93d3-46c8-a7ec-1d96aa2772c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:20.522179 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ed1a6-93d3-46c8-a7ec-1d96aa2772c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:20.522831 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81ed1a6-93d3-46c8-a7ec-1d96aa2772c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:20.524504 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81ed1a6-93d3-46c8-a7ec-1d96aa2772c2 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:20.534464 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 803/1600] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:20.537268 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-87ebf140-1bf9-439c-81f1-5acbbe0fb90d None None] MFA Rules not processed for user `b22838a8ad3949e59cdbaaa1801cd07f`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:20.539133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-87ebf140-1bf9-439c-81f1-5acbbe0fb90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:20.539294 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8817971-1d13-4650-a984-64ca5ed6987d None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:20.539575 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8817971-1d13-4650-a984-64ca5ed6987d None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:20.539852 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8817971-1d13-4650-a984-64ca5ed6987d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:20.568690 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-a8817971-1d13-4650-a984-64ca5ed6987d None None] MFA Rules not processed for user `20734b109838409192d9769a5a5de60b`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:20.570118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-a8817971-1d13-4650-a984-64ca5ed6987d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:20.600630 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:20.609605 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 798/1601] 10.222.0.76 () {58 vars in 1059 bytes} [Wed Aug 10 22:47:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:20.614202 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ebf140-1bf9-439c-81f1-5acbbe0fb90d None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:20.625519 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ebf140-1bf9-439c-81f1-5acbbe0fb90d None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:20.630190 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf412f4d-9258-4122-b24b-42692a7f394b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:20.630793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf412f4d-9258-4122-b24b-42692a7f394b admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:20.631074 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf412f4d-9258-4122-b24b-42692a7f394b admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:20.631347 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf412f4d-9258-4122-b24b-42692a7f394b admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:20.631982 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf412f4d-9258-4122-b24b-42692a7f394b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:20.634145 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf412f4d-9258-4122-b24b-42692a7f394b admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:20.638237 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 804/1602] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:20 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:20.639701 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db703322-5832-4140-93e2-c7028ed5f66f None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:20.639992 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db703322-5832-4140-93e2-c7028ed5f66f None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:20.640270 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db703322-5832-4140-93e2-c7028ed5f66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:20.647060 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 799/1603] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:20.648080 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf412f4d-9258-4122-b24b-42692a7f394b admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:20.658404 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf412f4d-9258-4122-b24b-42692a7f394b admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:20.663058 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223c216-2aa9-408f-89ba-e339fd03220e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:20.664175 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223c216-2aa9-408f-89ba-e339fd03220e admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:20.664452 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223c216-2aa9-408f-89ba-e339fd03220e admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:20.664742 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223c216-2aa9-408f-89ba-e339fd03220e admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:20.665454 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3223c216-2aa9-408f-89ba-e339fd03220e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:20.667370 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3223c216-2aa9-408f-89ba-e339fd03220e admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:20.668679 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-db703322-5832-4140-93e2-c7028ed5f66f None None] MFA Rules not processed for user `6e9467aadb8e4b1794a8fef711ea1f88`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:20.669921 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-db703322-5832-4140-93e2-c7028ed5f66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:20.700126 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 800/1604] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:47:20 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:20.700126 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223c216-2aa9-408f-89ba-e339fd03220e admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:20.736444 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 805/1605] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:20.737136 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db703322-5832-4140-93e2-c7028ed5f66f None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:20.748245 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db703322-5832-4140-93e2-c7028ed5f66f None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:20.764213 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223c216-2aa9-408f-89ba-e339fd03220e admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:20.768830 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a697e88-1d25-4388-ba86-1c12522f9545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:20.769469 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a697e88-1d25-4388-ba86-1c12522f9545 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:20.769772 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a697e88-1d25-4388-ba86-1c12522f9545 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:20.770043 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a697e88-1d25-4388-ba86-1c12522f9545 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:20.770673 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a697e88-1d25-4388-ba86-1c12522f9545 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:20.772263 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a697e88-1d25-4388-ba86-1c12522f9545 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:20.773827 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:20.784312 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 801/1606] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:20 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:20.785315 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a697e88-1d25-4388-ba86-1c12522f9545 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:20.795531 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a697e88-1d25-4388-ba86-1c12522f9545 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:20.800509 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05a278-1c6d-471f-add1-bbfa86fdb81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:20.801451 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05a278-1c6d-471f-add1-bbfa86fdb81b admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:20.801835 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05a278-1c6d-471f-add1-bbfa86fdb81b admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:20.802268 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05a278-1c6d-471f-add1-bbfa86fdb81b admin admin] PATH_INFO: `/v3/projects/aaae1e0f2ab8493cb12abbaf01f1833a/users/0d4dc87fae144ab9a40e1f047330168d/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:20.813457 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96406275-f64b-4536-bd47-4454cad8637d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:20.814132 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96406275-f64b-4536-bd47-4454cad8637d admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:20.814416 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96406275-f64b-4536-bd47-4454cad8637d admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:20.814694 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96406275-f64b-4536-bd47-4454cad8637d admin admin] PATH_INFO: `/v3/projects/a4e2567825f44d938dfbf7e043015367/users/12e6b0b71ecb4825aeab6735eb44e63e/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:20.821022 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba05a278-1c6d-471f-add1-bbfa86fdb81b admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaae1e0f2ab8493cb12abbaf01f1833a, user_id=0d4dc87fae144ab9a40e1f047330168d, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:20.823999 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba05a278-1c6d-471f-add1-bbfa86fdb81b admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:20.831388 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96406275-f64b-4536-bd47-4454cad8637d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4e2567825f44d938dfbf7e043015367, user_id=12e6b0b71ecb4825aeab6735eb44e63e, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:20.834218 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96406275-f64b-4536-bd47-4454cad8637d admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:20.837983 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 802/1607] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:47:20 2022] PUT /identity/v3/projects/aaae1e0f2ab8493cb12abbaf01f1833a/users/0d4dc87fae144ab9a40e1f047330168d/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:20.838861 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05a278-1c6d-471f-add1-bbfa86fdb81b admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:20.849611 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 806/1608] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:47:20 2022] PUT /identity/v3/projects/a4e2567825f44d938dfbf7e043015367/users/12e6b0b71ecb4825aeab6735eb44e63e/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:20.850440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96406275-f64b-4536-bd47-4454cad8637d admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:20.896591 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:20.914983 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96406275-f64b-4536-bd47-4454cad8637d admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:20.916182 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05a278-1c6d-471f-add1-bbfa86fdb81b admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:20.920576 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f60e258-c857-4f78-a18f-7380d66f7c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:20.921402 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f60e258-c857-4f78-a18f-7380d66f7c98 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:20.921682 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f60e258-c857-4f78-a18f-7380d66f7c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:20.921954 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f60e258-c857-4f78-a18f-7380d66f7c98 admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:20.922571 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f60e258-c857-4f78-a18f-7380d66f7c98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:20.924479 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f60e258-c857-4f78-a18f-7380d66f7c98 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:20.953469 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 807/1609] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:47:20 2022] POST /identity/v3/projects => generated 403 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:20.954441 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f60e258-c857-4f78-a18f-7380d66f7c98 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:20.985784 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5268854c-c130-41ed-88af-3205687b6011 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:20.986398 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5268854c-c130-41ed-88af-3205687b6011 service nova] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:20.986727 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5268854c-c130-41ed-88af-3205687b6011 service nova] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:20.986943 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5268854c-c130-41ed-88af-3205687b6011 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:21.016848 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f60e258-c857-4f78-a18f-7380d66f7c98 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:21.022335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdfe8ae-1c16-453b-a9cd-8f9aac7f6a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:21.022663 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdfe8ae-1c16-453b-a9cd-8f9aac7f6a21 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:21.023713 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdfe8ae-1c16-453b-a9cd-8f9aac7f6a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:21.024068 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdfe8ae-1c16-453b-a9cd-8f9aac7f6a21 admin admin] PATH_INFO: `/v3/projects/89c0b68704e04cdd84f27922e4da5e5d/users/da22fc66a1d14fa983463331de110258/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:21.039637 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdfe8ae-1c16-453b-a9cd-8f9aac7f6a21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89c0b68704e04cdd84f27922e4da5e5d, user_id=da22fc66a1d14fa983463331de110258, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:21.042409 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdfe8ae-1c16-453b-a9cd-8f9aac7f6a21 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:21.047285 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5268854c-c130-41ed-88af-3205687b6011 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:21.049102 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5268854c-c130-41ed-88af-3205687b6011 service nova] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:21.056229 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 808/1610] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:47:20 2022] PUT /identity/v3/projects/89c0b68704e04cdd84f27922e4da5e5d/users/da22fc66a1d14fa983463331de110258/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:21.057079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdfe8ae-1c16-453b-a9cd-8f9aac7f6a21 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:21.058790 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 803/1611] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:21.059621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5268854c-c130-41ed-88af-3205687b6011 service nova] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:21.071574 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5268854c-c130-41ed-88af-3205687b6011 service nova] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:21.077314 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:21.138474 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d7729-8be3-437e-a2b8-469c7583c2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:21.139191 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d7729-8be3-437e-a2b8-469c7583c2ce admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:21.139471 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d7729-8be3-437e-a2b8-469c7583c2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:21.139742 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d7729-8be3-437e-a2b8-469c7583c2ce admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:21.140365 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005d7729-8be3-437e-a2b8-469c7583c2ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:21.142318 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005d7729-8be3-437e-a2b8-469c7583c2ce admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:21.142547 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdfe8ae-1c16-453b-a9cd-8f9aac7f6a21 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:21.148923 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24e6b9a-60ac-46bb-ae20-2be3778175c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:21.149972 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e6b9a-60ac-46bb-ae20-2be3778175c8 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:21.150502 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e6b9a-60ac-46bb-ae20-2be3778175c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:21.150951 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e6b9a-60ac-46bb-ae20-2be3778175c8 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:21.151722 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24e6b9a-60ac-46bb-ae20-2be3778175c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:21.153896 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24e6b9a-60ac-46bb-ae20-2be3778175c8 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:21.172852 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 804/1612] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:47:21 2022] POST /identity/v3/projects => generated 371 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:21.174386 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d7729-8be3-437e-a2b8-469c7583c2ce admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:21.180910 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 809/1613] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:47:21 2022] POST /identity/v3/users => generated 288 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:21.183312 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e18d65-89c8-43cb-ad57-8a5e9f45b62f None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:21.183737 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e18d65-89c8-43cb-ad57-8a5e9f45b62f None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:21.184092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e18d65-89c8-43cb-ad57-8a5e9f45b62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:21.211568 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-30e18d65-89c8-43cb-ad57-8a5e9f45b62f None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:21.213007 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-30e18d65-89c8-43cb-ad57-8a5e9f45b62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:21.255005 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d7729-8be3-437e-a2b8-469c7583c2ce admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:21.255653 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:21.262720 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab7dae-733a-4cef-8adf-870a5a03f1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:21.263472 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab7dae-733a-4cef-8adf-870a5a03f1b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:21.263821 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab7dae-733a-4cef-8adf-870a5a03f1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:21.264174 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab7dae-733a-4cef-8adf-870a5a03f1b5 admin admin] PATH_INFO: `/v3/projects/b81a25303ced4ea4bf5351ec42192c13/users/405738ed219b4fdc8e3c07706a13e5a2/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:21.280889 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab7dae-733a-4cef-8adf-870a5a03f1b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b81a25303ced4ea4bf5351ec42192c13, user_id=405738ed219b4fdc8e3c07706a13e5a2, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:21.282267 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 810/1614] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:21.283807 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab7dae-733a-4cef-8adf-870a5a03f1b5 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:21.286554 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de619cbb-81f4-4934-a0ef-3fd50815b9f4 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:21.286910 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de619cbb-81f4-4934-a0ef-3fd50815b9f4 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:21.287350 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de619cbb-81f4-4934-a0ef-3fd50815b9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:21.309465 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 805/1615] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:47:21 2022] PUT /identity/v3/projects/b81a25303ced4ea4bf5351ec42192c13/users/405738ed219b4fdc8e3c07706a13e5a2/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:21.310457 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-de619cbb-81f4-4934-a0ef-3fd50815b9f4 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:21.310457 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-de619cbb-81f4-4934-a0ef-3fd50815b9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:21.311381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab7dae-733a-4cef-8adf-870a5a03f1b5 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:21.376338 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 811/1616] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:21.377740 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497064dd-d8f7-4b8d-b2da-bd00f7571ecc None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:21.378147 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497064dd-d8f7-4b8d-b2da-bd00f7571ecc None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:21.378499 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497064dd-d8f7-4b8d-b2da-bd00f7571ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:21.387056 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab7dae-733a-4cef-8adf-870a5a03f1b5 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:21.391598 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd645a-16de-4d5c-9ed1-f0f931722599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:21.392257 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd645a-16de-4d5c-9ed1-f0f931722599 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:21.392523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd645a-16de-4d5c-9ed1-f0f931722599 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:21.392810 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd645a-16de-4d5c-9ed1-f0f931722599 admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:21.393471 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd645a-16de-4d5c-9ed1-f0f931722599 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:21.395427 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd645a-16de-4d5c-9ed1-f0f931722599 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:21.405936 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-497064dd-d8f7-4b8d-b2da-bd00f7571ecc None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:21.407324 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-497064dd-d8f7-4b8d-b2da-bd00f7571ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:21.416914 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 812/1617] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:21.417843 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497064dd-d8f7-4b8d-b2da-bd00f7571ecc None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:21.422292 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 806/1618] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:47:21 2022] POST /identity/v3/users => generated 308 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:21.428372 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497064dd-d8f7-4b8d-b2da-bd00f7571ecc None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:21.429414 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd645a-16de-4d5c-9ed1-f0f931722599 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:21.433686 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1aae09-ad55-46a5-aa1f-d6c255f4954d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:21.434297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1aae09-ad55-46a5-aa1f-d6c255f4954d admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:21.434599 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1aae09-ad55-46a5-aa1f-d6c255f4954d admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:21.434850 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1aae09-ad55-46a5-aa1f-d6c255f4954d admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:21.435468 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1aae09-ad55-46a5-aa1f-d6c255f4954d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:21.435962 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:21.437522 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1aae09-ad55-46a5-aa1f-d6c255f4954d admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:21.440039 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd645a-16de-4d5c-9ed1-f0f931722599 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:21.446531 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2461c5-3b26-4ce1-8203-5ae8381d8d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:21.447240 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2461c5-3b26-4ce1-8203-5ae8381d8d4c admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:21.447582 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2461c5-3b26-4ce1-8203-5ae8381d8d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:21.447858 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2461c5-3b26-4ce1-8203-5ae8381d8d4c admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:21.448499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2461c5-3b26-4ce1-8203-5ae8381d8d4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:21.450286 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2461c5-3b26-4ce1-8203-5ae8381d8d4c admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:21.459937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 813/1619] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:47:21 2022] POST /identity/v3/users => generated 292 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:21.461716 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1aae09-ad55-46a5-aa1f-d6c255f4954d admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:21.462680 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 807/1620] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:21.464260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0846400-b4ee-44e7-875f-50547f59c7ac None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:21.464690 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0846400-b4ee-44e7-875f-50547f59c7ac None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:21.465082 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0846400-b4ee-44e7-875f-50547f59c7ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:21.484063 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-a0846400-b4ee-44e7-875f-50547f59c7ac None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:21.485304 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-a0846400-b4ee-44e7-875f-50547f59c7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:21.493622 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 808/1621] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:21.494610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434c61f-e7a6-4357-bda7-1e7436d712da None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:21.494892 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434c61f-e7a6-4357-bda7-1e7436d712da None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:21.495164 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434c61f-e7a6-4357-bda7-1e7436d712da None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:21.496405 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 809/1622] 10.222.0.76 () {54 vars in 912 bytes} [Wed Aug 10 22:47:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:47:21.497100 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434c61f-e7a6-4357-bda7-1e7436d712da None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:21.498734 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-0434c61f-e7a6-4357-bda7-1e7436d712da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:21.527308 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1aae09-ad55-46a5-aa1f-d6c255f4954d admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:21.531415 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855e411-5a7e-449b-8df4-2603c00ca40e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:21.532125 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855e411-5a7e-449b-8df4-2603c00ca40e service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:21.532469 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855e411-5a7e-449b-8df4-2603c00ca40e service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:21.532806 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855e411-5a7e-449b-8df4-2603c00ca40e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:21.543236 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1855e411-5a7e-449b-8df4-2603c00ca40e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:21.544667 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1855e411-5a7e-449b-8df4-2603c00ca40e service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:21.554900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 814/1623] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:21.555777 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855e411-5a7e-449b-8df4-2603c00ca40e service neutron] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:21.558408 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434c61f-e7a6-4357-bda7-1e7436d712da None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:21.562164 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a4881d-5fb8-4b40-aa9b-405abb127511 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:21.562791 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4881d-5fb8-4b40-aa9b-405abb127511 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:21.563072 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4881d-5fb8-4b40-aa9b-405abb127511 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:21.563340 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4881d-5fb8-4b40-aa9b-405abb127511 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:21.565237 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855e411-5a7e-449b-8df4-2603c00ca40e service neutron] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:21.570370 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f019b0c9-eddd-4536-964d-4eb6e83e7c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:21.570962 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019b0c9-eddd-4536-964d-4eb6e83e7c5e admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:21.571239 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019b0c9-eddd-4536-964d-4eb6e83e7c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:21.571512 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019b0c9-eddd-4536-964d-4eb6e83e7c5e admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:21.572682 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f019b0c9-eddd-4536-964d-4eb6e83e7c5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:21.573100 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a4881d-5fb8-4b40-aa9b-405abb127511 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:21.573793 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f019b0c9-eddd-4536-964d-4eb6e83e7c5e admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:21.574604 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a4881d-5fb8-4b40-aa9b-405abb127511 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:21.584276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 815/1624] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:21.585838 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 810/1625] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:21.586671 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a4881d-5fb8-4b40-aa9b-405abb127511 service neutron] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:21.588128 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f019b0c9-eddd-4536-964d-4eb6e83e7c5e admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:21.597569 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a4881d-5fb8-4b40-aa9b-405abb127511 service neutron] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:21.600297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f019b0c9-eddd-4536-964d-4eb6e83e7c5e admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:21.603961 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b448bb-15fd-42ec-b1a6-8445a6cf2283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:21.604667 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b448bb-15fd-42ec-b1a6-8445a6cf2283 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:21.605051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b448bb-15fd-42ec-b1a6-8445a6cf2283 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:21.605363 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b448bb-15fd-42ec-b1a6-8445a6cf2283 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:21.605821 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a43d82-a826-4f97-a777-6475edc67fad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:21.606130 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b448bb-15fd-42ec-b1a6-8445a6cf2283 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:21.606810 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a43d82-a826-4f97-a777-6475edc67fad service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:21.607374 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a43d82-a826-4f97-a777-6475edc67fad service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:21.607765 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a43d82-a826-4f97-a777-6475edc67fad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:21.607883 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b448bb-15fd-42ec-b1a6-8445a6cf2283 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:21.620087 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a43d82-a826-4f97-a777-6475edc67fad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:21.621871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a43d82-a826-4f97-a777-6475edc67fad service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:21.622024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 811/1626] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:21.622869 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b448bb-15fd-42ec-b1a6-8445a6cf2283 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:21.635729 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 816/1627] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:21.638150 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b448bb-15fd-42ec-b1a6-8445a6cf2283 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:21.638803 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd2444-d86b-45a4-a9f7-fd0614fe1a39 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:21.639404 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd2444-d86b-45a4-a9f7-fd0614fe1a39 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:21.640147 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd2444-d86b-45a4-a9f7-fd0614fe1a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:21.644583 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff758d-4afc-42dd-8262-66d0eb5b0206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:21.645553 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff758d-4afc-42dd-8262-66d0eb5b0206 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:21.645929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff758d-4afc-42dd-8262-66d0eb5b0206 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:21.646400 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff758d-4afc-42dd-8262-66d0eb5b0206 admin admin] PATH_INFO: `/v3/projects/b24b2bcb760f4e658ab6f2e7ff57847b/users/81cfc1a8d5ab497fa4159ca6c8736cb7/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:21.666883 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff758d-4afc-42dd-8262-66d0eb5b0206 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b24b2bcb760f4e658ab6f2e7ff57847b, user_id=81cfc1a8d5ab497fa4159ca6c8736cb7, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:21.670229 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff758d-4afc-42dd-8262-66d0eb5b0206 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:21.676062 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-86dd2444-d86b-45a4-a9f7-fd0614fe1a39 None None] MFA Rules not processed for user `b22838a8ad3949e59cdbaaa1801cd07f`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:21.677937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-86dd2444-d86b-45a4-a9f7-fd0614fe1a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:21.692063 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 812/1628] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:47:21 2022] PUT /identity/v3/projects/b24b2bcb760f4e658ab6f2e7ff57847b/users/81cfc1a8d5ab497fa4159ca6c8736cb7/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:21.692063 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff758d-4afc-42dd-8262-66d0eb5b0206 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:21.721366 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:21.773355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 817/1629] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 10 22:47:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:21.775429 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff758d-4afc-42dd-8262-66d0eb5b0206 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:21.775795 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd2444-d86b-45a4-a9f7-fd0614fe1a39 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:21.783901 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02943989-813b-4d6f-9b83-8a3716cdc1c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:21.785785 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02943989-813b-4d6f-9b83-8a3716cdc1c0 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:21.786917 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02943989-813b-4d6f-9b83-8a3716cdc1c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:21.787253 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02943989-813b-4d6f-9b83-8a3716cdc1c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:21.897126 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02943989-813b-4d6f-9b83-8a3716cdc1c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:21.899471 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02943989-813b-4d6f-9b83-8a3716cdc1c0 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:21.903876 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd2444-d86b-45a4-a9f7-fd0614fe1a39 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:21.913003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c18d8fb-55b4-4257-a5d3-802d79b3c4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:21.916106 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18d8fb-55b4-4257-a5d3-802d79b3c4b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:21.916560 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18d8fb-55b4-4257-a5d3-802d79b3c4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:21.917454 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18d8fb-55b4-4257-a5d3-802d79b3c4b7 admin admin] PATH_INFO: `/v3/projects/242c0cd9ce454a26b8409ffc5c25fcf6/users/dcb322c4321f4abd914b05f7c2d4636d/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:21.919193 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 813/1630] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:21.923877 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02943989-813b-4d6f-9b83-8a3716cdc1c0 service neutron] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:21.937617 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:21.947777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02943989-813b-4d6f-9b83-8a3716cdc1c0 service neutron] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:21.957068 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c18d8fb-55b4-4257-a5d3-802d79b3c4b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=242c0cd9ce454a26b8409ffc5c25fcf6, user_id=dcb322c4321f4abd914b05f7c2d4636d, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:21.959216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f8d48-48c9-4c17-ab89-de6bba9cc6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:21.961369 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f8d48-48c9-4c17-ab89-de6bba9cc6f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:21.961834 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f8d48-48c9-4c17-ab89-de6bba9cc6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:21.962406 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f8d48-48c9-4c17-ab89-de6bba9cc6f7 admin admin] PATH_INFO: `/v3/projects/b1f28560c6db49afa2cdfbf5dc583d1d/users/10b96a3cc2b249a3bf3b9163a364d01f/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:21.964046 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c18d8fb-55b4-4257-a5d3-802d79b3c4b7 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:21.985823 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3f8d48-48c9-4c17-ab89-de6bba9cc6f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1f28560c6db49afa2cdfbf5dc583d1d, user_id=10b96a3cc2b249a3bf3b9163a364d01f, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:21.988973 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3f8d48-48c9-4c17-ab89-de6bba9cc6f7 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:21.990653 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 818/1631] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:47:21 2022] PUT /identity/v3/projects/242c0cd9ce454a26b8409ffc5c25fcf6/users/dcb322c4321f4abd914b05f7c2d4636d/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:21.993118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c18d8fb-55b4-4257-a5d3-802d79b3c4b7 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:22.014200 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 814/1632] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:47:21 2022] PUT /identity/v3/projects/b1f28560c6db49afa2cdfbf5dc583d1d/users/10b96a3cc2b249a3bf3b9163a364d01f/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:22.014200 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f8d48-48c9-4c17-ab89-de6bba9cc6f7 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:22.080568 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c18d8fb-55b4-4257-a5d3-802d79b3c4b7 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:22.090262 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f8d48-48c9-4c17-ab89-de6bba9cc6f7 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:22.095050 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc26cb7-41be-495d-8824-2e2be3f7f546 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:22.095681 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc26cb7-41be-495d-8824-2e2be3f7f546 service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:22.095961 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc26cb7-41be-495d-8824-2e2be3f7f546 service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:22.096515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc26cb7-41be-495d-8824-2e2be3f7f546 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:22.162064 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731ac43-4657-4bf0-b62a-9b47a7a1562d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:22.163295 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731ac43-4657-4bf0-b62a-9b47a7a1562d admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:22.164904 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731ac43-4657-4bf0-b62a-9b47a7a1562d admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:22.175253 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731ac43-4657-4bf0-b62a-9b47a7a1562d admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:22.177194 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731ac43-4657-4bf0-b62a-9b47a7a1562d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:22.180693 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731ac43-4657-4bf0-b62a-9b47a7a1562d admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:22.183933 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:22.209811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 819/1633] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:22.211283 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731ac43-4657-4bf0-b62a-9b47a7a1562d admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:22.225391 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731ac43-4657-4bf0-b62a-9b47a7a1562d admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:22.232641 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc26cb7-41be-495d-8824-2e2be3f7f546 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:22.233929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8e346-e675-4d43-b5e2-f4999d691f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:22.234407 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc26cb7-41be-495d-8824-2e2be3f7f546 service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:22.234604 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8e346-e675-4d43-b5e2-f4999d691f9a admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:22.235035 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8e346-e675-4d43-b5e2-f4999d691f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:22.235459 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8e346-e675-4d43-b5e2-f4999d691f9a admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:22.236304 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e8e346-e675-4d43-b5e2-f4999d691f9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:22.238484 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e8e346-e675-4d43-b5e2-f4999d691f9a admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:22.260132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 815/1634] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:22.264904 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 820/1635] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:22.265483 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc26cb7-41be-495d-8824-2e2be3f7f546 service cinder] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:22.266396 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8e346-e675-4d43-b5e2-f4999d691f9a admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:22.283292 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8e346-e675-4d43-b5e2-f4999d691f9a admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:22.292177 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dec441-a72b-46eb-82d1-3dd821afb6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:22.294239 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dec441-a72b-46eb-82d1-3dd821afb6e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:22.294725 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dec441-a72b-46eb-82d1-3dd821afb6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:22.295242 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dec441-a72b-46eb-82d1-3dd821afb6e7 admin admin] PATH_INFO: `/v3/projects/b24b2bcb760f4e658ab6f2e7ff57847b/users/81cfc1a8d5ab497fa4159ca6c8736cb7/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:22.295856 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc26cb7-41be-495d-8824-2e2be3f7f546 service cinder] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:22.299208 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dec441-a72b-46eb-82d1-3dd821afb6e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b24b2bcb760f4e658ab6f2e7ff57847b, user_id=81cfc1a8d5ab497fa4159ca6c8736cb7, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:22.304230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dec441-a72b-46eb-82d1-3dd821afb6e7 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:22.308910 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e08048-baea-4b46-b712-481bafe043f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:22.309564 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e08048-baea-4b46-b712-481bafe043f1 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:22.310978 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e08048-baea-4b46-b712-481bafe043f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:22.311331 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e08048-baea-4b46-b712-481bafe043f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:22.312783 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e08048-baea-4b46-b712-481bafe043f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:22.316133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e08048-baea-4b46-b712-481bafe043f1 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:22.321680 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 821/1636] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:47:22 2022] PUT /identity/v3/projects/b24b2bcb760f4e658ab6f2e7ff57847b/users/81cfc1a8d5ab497fa4159ca6c8736cb7/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:22.323413 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dec441-a72b-46eb-82d1-3dd821afb6e7 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:22.349007 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 816/1637] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:22.354883 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ec2b7-fe90-4209-9fe8-21111526a7cf None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:22.355420 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ec2b7-fe90-4209-9fe8-21111526a7cf None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:22.356259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ec2b7-fe90-4209-9fe8-21111526a7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:22.450981 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dec441-a72b-46eb-82d1-3dd821afb6e7 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:22.454781 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-1f7ec2b7-fe90-4209-9fe8-21111526a7cf None None] MFA Rules not processed for user `81cfc1a8d5ab497fa4159ca6c8736cb7`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:22.461243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-1f7ec2b7-fe90-4209-9fe8-21111526a7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:22.466747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd51a6-a7ec-4e01-9115-6523ca28a2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:22.468439 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd51a6-a7ec-4e01-9115-6523ca28a2de admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:22.468816 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd51a6-a7ec-4e01-9115-6523ca28a2de admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:22.469203 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd51a6-a7ec-4e01-9115-6523ca28a2de admin admin] PATH_INFO: `/v3/projects/242c0cd9ce454a26b8409ffc5c25fcf6/users/dcb322c4321f4abd914b05f7c2d4636d/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:22.472079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebd51a6-a7ec-4e01-9115-6523ca28a2de admin admin] RBAC: Authorizing `identity:create_grant(project_id=242c0cd9ce454a26b8409ffc5c25fcf6, user_id=dcb322c4321f4abd914b05f7c2d4636d, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:22.480611 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebd51a6-a7ec-4e01-9115-6523ca28a2de admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:22.499874 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 822/1638] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:47:22 2022] PUT /identity/v3/projects/242c0cd9ce454a26b8409ffc5c25fcf6/users/dcb322c4321f4abd914b05f7c2d4636d/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:22.510293 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd51a6-a7ec-4e01-9115-6523ca28a2de admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:22.598470 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd51a6-a7ec-4e01-9115-6523ca28a2de admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:22.601810 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:22.606654 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9cfe05-5b8d-4001-a94a-2ca22e146d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:22.608067 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9cfe05-5b8d-4001-a94a-2ca22e146d1f admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:22.608765 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9cfe05-5b8d-4001-a94a-2ca22e146d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:22.609333 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9cfe05-5b8d-4001-a94a-2ca22e146d1f admin admin] PATH_INFO: `/v3/projects/b1f28560c6db49afa2cdfbf5dc583d1d/users/10b96a3cc2b249a3bf3b9163a364d01f/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:22.614225 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9cfe05-5b8d-4001-a94a-2ca22e146d1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1f28560c6db49afa2cdfbf5dc583d1d, user_id=10b96a3cc2b249a3bf3b9163a364d01f, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:22.619836 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9cfe05-5b8d-4001-a94a-2ca22e146d1f admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:22.625730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 817/1639] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:22 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:22.626935 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610caa10-ddac-42e1-83c1-660892c08e6a None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:22.627223 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610caa10-ddac-42e1-83c1-660892c08e6a None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:22.627504 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610caa10-ddac-42e1-83c1-660892c08e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:22.651980 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 823/1640] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:47:22 2022] PUT /identity/v3/projects/b1f28560c6db49afa2cdfbf5dc583d1d/users/10b96a3cc2b249a3bf3b9163a364d01f/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:22.653142 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9cfe05-5b8d-4001-a94a-2ca22e146d1f admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:22.699402 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-610caa10-ddac-42e1-83c1-660892c08e6a None None] MFA Rules not processed for user `dcb322c4321f4abd914b05f7c2d4636d`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:22.700705 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-610caa10-ddac-42e1-83c1-660892c08e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:22.788960 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9cfe05-5b8d-4001-a94a-2ca22e146d1f admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:22.793719 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9feec2-e5d5-43b3-b404-d774ffa00d1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:22.794071 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9feec2-e5d5-43b3-b404-d774ffa00d1f service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:22.795120 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9feec2-e5d5-43b3-b404-d774ffa00d1f service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:22.795257 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9feec2-e5d5-43b3-b404-d774ffa00d1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:22.811203 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:22.876774 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9feec2-e5d5-43b3-b404-d774ffa00d1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:22.880105 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9feec2-e5d5-43b3-b404-d774ffa00d1f service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:22.887526 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 818/1641] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:22 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:22.889989 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492897f-e983-43c3-b6f8-e05764a64c6f None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:22.890760 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492897f-e983-43c3-b6f8-e05764a64c6f None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:22.891214 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492897f-e983-43c3-b6f8-e05764a64c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:22.895779 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 824/1642] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:22 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:22.923752 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-7492897f-e983-43c3-b6f8-e05764a64c6f None None] MFA Rules not processed for user `10b96a3cc2b249a3bf3b9163a364d01f`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:22.924972 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-7492897f-e983-43c3-b6f8-e05764a64c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:22.978567 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9feec2-e5d5-43b3-b404-d774ffa00d1f service cinder] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:22.991419 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9feec2-e5d5-43b3-b404-d774ffa00d1f service cinder] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:22.998931 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b715a9-8cc4-4f19-b449-03a3d1e8f667 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:23.001230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b715a9-8cc4-4f19-b449-03a3d1e8f667 service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:23.001618 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b715a9-8cc4-4f19-b449-03a3d1e8f667 service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:23.001892 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b715a9-8cc4-4f19-b449-03a3d1e8f667 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:23.025161 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 819/1643] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:22 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:23.037491 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b715a9-8cc4-4f19-b449-03a3d1e8f667 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:23.039736 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b715a9-8cc4-4f19-b449-03a3d1e8f667 service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:23.053145 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 825/1644] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:22 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:23.255315 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492897f-e983-43c3-b6f8-e05764a64c6f None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:23.267853 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492897f-e983-43c3-b6f8-e05764a64c6f None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:23.272158 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a01cb2-adab-478f-bceb-0a8d0ebd1b17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:23.273631 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a01cb2-adab-478f-bceb-0a8d0ebd1b17 service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:23.275533 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a01cb2-adab-478f-bceb-0a8d0ebd1b17 service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:23.275675 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a01cb2-adab-478f-bceb-0a8d0ebd1b17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:23.314397 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a01cb2-adab-478f-bceb-0a8d0ebd1b17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:23.318157 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a01cb2-adab-478f-bceb-0a8d0ebd1b17 service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:23.347077 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 820/1645] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:23 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:23.430240 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b715a9-8cc4-4f19-b449-03a3d1e8f667 service cinder] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:23.458665 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b715a9-8cc4-4f19-b449-03a3d1e8f667 service cinder] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:23.467342 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a9babd-21a4-4f27-afeb-060f85dfc927 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:23.468547 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9babd-21a4-4f27-afeb-060f85dfc927 service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:23.468891 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9babd-21a4-4f27-afeb-060f85dfc927 service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:23.469180 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9babd-21a4-4f27-afeb-060f85dfc927 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:23.614953 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a9babd-21a4-4f27-afeb-060f85dfc927 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:23.617996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a9babd-21a4-4f27-afeb-060f85dfc927 service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:23.640791 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 826/1646] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:23.841408 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a01cb2-adab-478f-bceb-0a8d0ebd1b17 service cinder] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:23.992337 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:24.015979 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a01cb2-adab-478f-bceb-0a8d0ebd1b17 service cinder] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:24.027243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c1dd36-3e85-4a45-95f5-f6982d6d2434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:24.027898 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c1dd36-3e85-4a45-95f5-f6982d6d2434 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:24.030873 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c1dd36-3e85-4a45-95f5-f6982d6d2434 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:24.031173 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c1dd36-3e85-4a45-95f5-f6982d6d2434 admin admin] PATH_INFO: `/v3/users/20734b109838409192d9769a5a5de60b` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:24.032932 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c1dd36-3e85-4a45-95f5-f6982d6d2434 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20734b109838409192d9769a5a5de60b)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:24.039159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c1dd36-3e85-4a45-95f5-f6982d6d2434 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:24.158946 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565ccb6-3c90-4cbd-9893-023d347466eb None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:24.159446 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-88c1dd36-3e85-4a45-95f5-f6982d6d2434 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20734b109838409192d9769a5a5de60b'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:24.159708 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565ccb6-3c90-4cbd-9893-023d347466eb None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:24.160101 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565ccb6-3c90-4cbd-9893-023d347466eb None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:24.161625 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 827/1647] 10.222.0.76 () {54 vars in 962 bytes} [Wed Aug 10 22:47:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:47:24.171281 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbdf3fd-c2c9-43bc-b428-ccf232be3446 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:24.171873 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbdf3fd-c2c9-43bc-b428-ccf232be3446 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:24.172261 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbdf3fd-c2c9-43bc-b428-ccf232be3446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:24.182987 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-88c1dd36-3e85-4a45-95f5-f6982d6d2434 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20734b109838409192d9769a5a5de60b'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:24.219495 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-88c1dd36-3e85-4a45-95f5-f6982d6d2434 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20734b109838409192d9769a5a5de60b'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:24.221623 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-88c1dd36-3e85-4a45-95f5-f6982d6d2434 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20734b109838409192d9769a5a5de60b'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:24.222262 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-88c1dd36-3e85-4a45-95f5-f6982d6d2434 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20734b109838409192d9769a5a5de60b'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:24.230298 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-2fbdf3fd-c2c9-43bc-b428-ccf232be3446 None None] MFA Rules not processed for user `b22838a8ad3949e59cdbaaa1801cd07f`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:24.234477 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-2fbdf3fd-c2c9-43bc-b428-ccf232be3446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:24.249916 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 821/1648] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:47:23 2022] DELETE /identity/v3/users/20734b109838409192d9769a5a5de60b => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:24.257383 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060bac5d-6e31-4359-baae-c846dcb7d724 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:24.257383 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060bac5d-6e31-4359-baae-c846dcb7d724 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:24.257383 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060bac5d-6e31-4359-baae-c846dcb7d724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:24.298520 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-060bac5d-6e31-4359-baae-c846dcb7d724 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:24.299955 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-060bac5d-6e31-4359-baae-c846dcb7d724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:24.385572 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 828/1649] 10.222.0.76 () {58 vars in 1059 bytes} [Wed Aug 10 22:47:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:24.386959 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc1749-ed48-47ed-8a3a-36aaab4e1cc8 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:24.387251 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc1749-ed48-47ed-8a3a-36aaab4e1cc8 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:24.387783 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc1749-ed48-47ed-8a3a-36aaab4e1cc8 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:24.402410 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 829/1650] 10.222.0.76 () {54 vars in 912 bytes} [Wed Aug 10 22:47:24 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:47:24.403807 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dde6ac-8575-4aed-bf05-30096a3122dc None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:24.404093 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dde6ac-8575-4aed-bf05-30096a3122dc None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:24.404370 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dde6ac-8575-4aed-bf05-30096a3122dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:24.439827 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-57dde6ac-8575-4aed-bf05-30096a3122dc None None] MFA Rules not processed for user `b22838a8ad3949e59cdbaaa1801cd07f`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:24.443585 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-57dde6ac-8575-4aed-bf05-30096a3122dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:24.450652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 822/1651] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:24.466982 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060bac5d-6e31-4359-baae-c846dcb7d724 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:24.469169 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-060bac5d-6e31-4359-baae-c846dcb7d724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:24.491968 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:24.556678 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 830/1652] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 10 22:47:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:24.557736 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dde6ac-8575-4aed-bf05-30096a3122dc None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:24.564449 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-57dde6ac-8575-4aed-bf05-30096a3122dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:24.587887 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dde6ac-8575-4aed-bf05-30096a3122dc None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:24.595647 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547faa0-0194-4008-8aaf-27ad8156e37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:24.595911 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547faa0-0194-4008-8aaf-27ad8156e37f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:24.596013 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547faa0-0194-4008-8aaf-27ad8156e37f admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:24.596324 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547faa0-0194-4008-8aaf-27ad8156e37f admin admin] PATH_INFO: `/v3/users/6e9467aadb8e4b1794a8fef711ea1f88` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:24.598740 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6547faa0-0194-4008-8aaf-27ad8156e37f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e9467aadb8e4b1794a8fef711ea1f88)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:24.599802 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: WARNING py.warnings [None req-6547faa0-0194-4008-8aaf-27ad8156e37f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 22:47:24.599802 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: warnings.warn(msg) Aug 10 22:47:24.599802 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: Aug 10 22:47:24.601065 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6547faa0-0194-4008-8aaf-27ad8156e37f admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:24.601966 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060bac5d-6e31-4359-baae-c846dcb7d724 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:24.608292 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d9984-9eef-4b4c-9add-dbfae54337e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:24.609320 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d9984-9eef-4b4c-9add-dbfae54337e9 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:24.609722 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d9984-9eef-4b4c-9add-dbfae54337e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:24.609894 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d9984-9eef-4b4c-9add-dbfae54337e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:24.633693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63d9984-9eef-4b4c-9add-dbfae54337e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:24.635321 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63d9984-9eef-4b4c-9add-dbfae54337e9 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:24.659078 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 823/1653] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:24.660334 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d9984-9eef-4b4c-9add-dbfae54337e9 service neutron] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:24.662108 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-c63d9984-9eef-4b4c-9add-dbfae54337e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:24.698997 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-6547faa0-0194-4008-8aaf-27ad8156e37f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e9467aadb8e4b1794a8fef711ea1f88'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:24.711177 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:24.720614 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-6547faa0-0194-4008-8aaf-27ad8156e37f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e9467aadb8e4b1794a8fef711ea1f88'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:24.748477 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-6547faa0-0194-4008-8aaf-27ad8156e37f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e9467aadb8e4b1794a8fef711ea1f88'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:24.750005 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-6547faa0-0194-4008-8aaf-27ad8156e37f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e9467aadb8e4b1794a8fef711ea1f88'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:24.750853 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-6547faa0-0194-4008-8aaf-27ad8156e37f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e9467aadb8e4b1794a8fef711ea1f88'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:24.801416 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d9984-9eef-4b4c-9add-dbfae54337e9 service neutron] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:24.804507 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 831/1654] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:47:24 2022] DELETE /identity/v3/users/6e9467aadb8e4b1794a8fef711ea1f88 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:24.810562 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127317b5-cc1b-4cd9-87c3-ba8f4360af50 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:24.811100 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127317b5-cc1b-4cd9-87c3-ba8f4360af50 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:24.811530 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127317b5-cc1b-4cd9-87c3-ba8f4360af50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:24.847703 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-127317b5-cc1b-4cd9-87c3-ba8f4360af50 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:24.853152 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-127317b5-cc1b-4cd9-87c3-ba8f4360af50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:24.923265 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9c010-9eaf-4ae6-b68e-b66b56cc7595 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:24.923959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9c010-9eaf-4ae6-b68e-b66b56cc7595 service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:24.924260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9c010-9eaf-4ae6-b68e-b66b56cc7595 service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:24.924528 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9c010-9eaf-4ae6-b68e-b66b56cc7595 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:24.926680 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-fbf9c010-9eaf-4ae6-b68e-b66b56cc7595 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:24.974312 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 832/1655] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:24.992240 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127317b5-cc1b-4cd9-87c3-ba8f4360af50 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:24.996005 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-127317b5-cc1b-4cd9-87c3-ba8f4360af50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:25.059694 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:25.111222 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127317b5-cc1b-4cd9-87c3-ba8f4360af50 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:25.115911 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2898a43-51e9-4211-8c77-3d602604ff39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:25.118220 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2898a43-51e9-4211-8c77-3d602604ff39 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:25.118220 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2898a43-51e9-4211-8c77-3d602604ff39 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:25.118220 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2898a43-51e9-4211-8c77-3d602604ff39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:25.153587 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2898a43-51e9-4211-8c77-3d602604ff39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:25.157719 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2898a43-51e9-4211-8c77-3d602604ff39 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:25.186898 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9c010-9eaf-4ae6-b68e-b66b56cc7595 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:25.187462 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 833/1656] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:25.188388 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9c010-9eaf-4ae6-b68e-b66b56cc7595 service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:25.204011 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 824/1657] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:26.016155 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2898a43-51e9-4211-8c77-3d602604ff39 service neutron] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:26.017775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-a2898a43-51e9-4211-8c77-3d602604ff39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:26.031903 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2898a43-51e9-4211-8c77-3d602604ff39 service neutron] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:26.038804 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b25e79f-2630-4322-a494-070649b8faf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:26.039477 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b25e79f-2630-4322-a494-070649b8faf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:26.039756 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b25e79f-2630-4322-a494-070649b8faf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:26.040126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b25e79f-2630-4322-a494-070649b8faf1 admin admin] PATH_INFO: `/v3/projects/5335171be4fe4b9189674285700e696b` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:26.041574 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b25e79f-2630-4322-a494-070649b8faf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5335171be4fe4b9189674285700e696b)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:26.044681 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b25e79f-2630-4322-a494-070649b8faf1 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:26.142607 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-5b25e79f-2630-4322-a494-070649b8faf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5335171be4fe4b9189674285700e696b', 'request_id': 'req-5b25e79f-2630-4322-a494-070649b8faf1'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:26.158198 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-5b25e79f-2630-4322-a494-070649b8faf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5335171be4fe4b9189674285700e696b', 'request_id': 'req-5b25e79f-2630-4322-a494-070649b8faf1'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:26.166968 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-5b25e79f-2630-4322-a494-070649b8faf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5335171be4fe4b9189674285700e696b', 'request_id': 'req-5b25e79f-2630-4322-a494-070649b8faf1'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:26.188121 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-5b25e79f-2630-4322-a494-070649b8faf1 admin admin] The token cache is being invalidate because project 5335171be4fe4b9189674285700e696b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:47:26.188418 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-5b25e79f-2630-4322-a494-070649b8faf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:26.189107 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-5b25e79f-2630-4322-a494-070649b8faf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:26.248453 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 834/1658] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:47:26 2022] DELETE /identity/v3/projects/5335171be4fe4b9189674285700e696b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:26.248909 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9c010-9eaf-4ae6-b68e-b66b56cc7595 service cinder] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:26.283051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9c010-9eaf-4ae6-b68e-b66b56cc7595 service cinder] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:26.402532 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf97889-a236-46e4-8a3c-af1ca0ed3f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:26.403186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf97889-a236-46e4-8a3c-af1ca0ed3f1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:26.403593 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf97889-a236-46e4-8a3c-af1ca0ed3f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:26.404138 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf97889-a236-46e4-8a3c-af1ca0ed3f1a admin admin] PATH_INFO: `/v3/users/dcb322c4321f4abd914b05f7c2d4636d` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:26.405404 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf97889-a236-46e4-8a3c-af1ca0ed3f1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcb322c4321f4abd914b05f7c2d4636d)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:26.407494 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf97889-a236-46e4-8a3c-af1ca0ed3f1a admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:26.439275 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:26.515912 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-fbf97889-a236-46e4-8a3c-af1ca0ed3f1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcb322c4321f4abd914b05f7c2d4636d'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:26.526562 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-fbf97889-a236-46e4-8a3c-af1ca0ed3f1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcb322c4321f4abd914b05f7c2d4636d'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:26.558086 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-fbf97889-a236-46e4-8a3c-af1ca0ed3f1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcb322c4321f4abd914b05f7c2d4636d'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:26.559060 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-fbf97889-a236-46e4-8a3c-af1ca0ed3f1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcb322c4321f4abd914b05f7c2d4636d'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:26.560678 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-fbf97889-a236-46e4-8a3c-af1ca0ed3f1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcb322c4321f4abd914b05f7c2d4636d'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:26.593561 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 825/1659] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:47:26 2022] DELETE /identity/v3/users/dcb322c4321f4abd914b05f7c2d4636d => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:26.639021 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b25e79f-2630-4322-a494-070649b8faf1 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:26.640650 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-5b25e79f-2630-4322-a494-070649b8faf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:26.664775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b25e79f-2630-4322-a494-070649b8faf1 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:26.794083 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf2a26d-9703-4446-87ba-3b46c25d98ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:26.795352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2a26d-9703-4446-87ba-3b46c25d98ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:26.795352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2a26d-9703-4446-87ba-3b46c25d98ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:26.795696 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2a26d-9703-4446-87ba-3b46c25d98ae admin admin] PATH_INFO: `/v3/projects/2daad69c6ae0460db7ac5c9df6efd7a0` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:26.797139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf2a26d-9703-4446-87ba-3b46c25d98ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=2daad69c6ae0460db7ac5c9df6efd7a0)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:26.801058 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf2a26d-9703-4446-87ba-3b46c25d98ae admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:26.928445 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-3cf2a26d-9703-4446-87ba-3b46c25d98ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2daad69c6ae0460db7ac5c9df6efd7a0', 'request_id': 'req-3cf2a26d-9703-4446-87ba-3b46c25d98ae'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:26.938356 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-3cf2a26d-9703-4446-87ba-3b46c25d98ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2daad69c6ae0460db7ac5c9df6efd7a0', 'request_id': 'req-3cf2a26d-9703-4446-87ba-3b46c25d98ae'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:26.941250 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:26.981359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-3cf2a26d-9703-4446-87ba-3b46c25d98ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2daad69c6ae0460db7ac5c9df6efd7a0', 'request_id': 'req-3cf2a26d-9703-4446-87ba-3b46c25d98ae'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:27.074309 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-3cf2a26d-9703-4446-87ba-3b46c25d98ae admin admin] The token cache is being invalidate because project 2daad69c6ae0460db7ac5c9df6efd7a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:47:27.076196 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-3cf2a26d-9703-4446-87ba-3b46c25d98ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:27.077670 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-3cf2a26d-9703-4446-87ba-3b46c25d98ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:27.086591 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 835/1660] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:47:26 2022] DELETE /identity/v3/projects/2daad69c6ae0460db7ac5c9df6efd7a0 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:27.088292 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf97889-a236-46e4-8a3c-af1ca0ed3f1a admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:27.090021 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-fbf97889-a236-46e4-8a3c-af1ca0ed3f1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:27.120278 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf97889-a236-46e4-8a3c-af1ca0ed3f1a admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:27.292352 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc6c9d-dbff-4f5c-a6fc-398e6388b610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:27.293231 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc6c9d-dbff-4f5c-a6fc-398e6388b610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:27.293547 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc6c9d-dbff-4f5c-a6fc-398e6388b610 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:27.293875 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc6c9d-dbff-4f5c-a6fc-398e6388b610 admin admin] PATH_INFO: `/v3/users/10b96a3cc2b249a3bf3b9163a364d01f` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:27.295470 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc6c9d-dbff-4f5c-a6fc-398e6388b610 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10b96a3cc2b249a3bf3b9163a364d01f)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:27.307519 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc6c9d-dbff-4f5c-a6fc-398e6388b610 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:27.387977 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf2a26d-9703-4446-87ba-3b46c25d98ae admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:27.390076 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-3cf2a26d-9703-4446-87ba-3b46c25d98ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:27.425722 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf2a26d-9703-4446-87ba-3b46c25d98ae admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:27.442820 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a39f5d-7d3f-4675-aa0a-bc7c40c99f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:27.447289 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a39f5d-7d3f-4675-aa0a-bc7c40c99f26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:27.448093 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a39f5d-7d3f-4675-aa0a-bc7c40c99f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:27.450961 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a39f5d-7d3f-4675-aa0a-bc7c40c99f26 admin admin] PATH_INFO: `/v3/users/ffd10984bde54bb586ecc86386ae3b19` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:27.454655 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a39f5d-7d3f-4675-aa0a-bc7c40c99f26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffd10984bde54bb586ecc86386ae3b19)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:27.465109 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a39f5d-7d3f-4675-aa0a-bc7c40c99f26 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:27.473397 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-40fc6c9d-dbff-4f5c-a6fc-398e6388b610 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10b96a3cc2b249a3bf3b9163a364d01f'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:27.506042 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-40fc6c9d-dbff-4f5c-a6fc-398e6388b610 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10b96a3cc2b249a3bf3b9163a364d01f'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:27.547114 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-40fc6c9d-dbff-4f5c-a6fc-398e6388b610 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10b96a3cc2b249a3bf3b9163a364d01f'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:27.549229 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-40fc6c9d-dbff-4f5c-a6fc-398e6388b610 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10b96a3cc2b249a3bf3b9163a364d01f'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:27.551912 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-40fc6c9d-dbff-4f5c-a6fc-398e6388b610 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10b96a3cc2b249a3bf3b9163a364d01f'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:27.604501 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 826/1661] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:47:27 2022] DELETE /identity/v3/users/10b96a3cc2b249a3bf3b9163a364d01f => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:27.622308 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-c0a39f5d-7d3f-4675-aa0a-bc7c40c99f26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffd10984bde54bb586ecc86386ae3b19'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:27.635982 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-c0a39f5d-7d3f-4675-aa0a-bc7c40c99f26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffd10984bde54bb586ecc86386ae3b19'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:27.652715 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-c0a39f5d-7d3f-4675-aa0a-bc7c40c99f26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffd10984bde54bb586ecc86386ae3b19'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:27.654504 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-c0a39f5d-7d3f-4675-aa0a-bc7c40c99f26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffd10984bde54bb586ecc86386ae3b19'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:27.655169 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-c0a39f5d-7d3f-4675-aa0a-bc7c40c99f26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffd10984bde54bb586ecc86386ae3b19'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:27.736977 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 836/1662] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:47:27 2022] DELETE /identity/v3/users/ffd10984bde54bb586ecc86386ae3b19 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:47:27.743313 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5fb95-3c57-455b-9f44-200edde6ba0c None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:27.746254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5fb95-3c57-455b-9f44-200edde6ba0c None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:27.746663 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5fb95-3c57-455b-9f44-200edde6ba0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:27.822705 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-28a5fb95-3c57-455b-9f44-200edde6ba0c None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:27.823968 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-28a5fb95-3c57-455b-9f44-200edde6ba0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:27.841269 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:28.025460 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 827/1663] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:28.167112 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a39f5d-7d3f-4675-aa0a-bc7c40c99f26 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:28.169808 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-c0a39f5d-7d3f-4675-aa0a-bc7c40c99f26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:28.307686 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5fb95-3c57-455b-9f44-200edde6ba0c None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:28.309144 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-28a5fb95-3c57-455b-9f44-200edde6ba0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:28.339633 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5fb95-3c57-455b-9f44-200edde6ba0c None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:28.358686 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a39f5d-7d3f-4675-aa0a-bc7c40c99f26 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:28.362199 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9998b-5872-4d8c-b793-15e91df14ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:28.363924 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9998b-5872-4d8c-b793-15e91df14ee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:28.364233 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9998b-5872-4d8c-b793-15e91df14ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:28.364514 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9998b-5872-4d8c-b793-15e91df14ee1 admin admin] PATH_INFO: `/v3/projects/242c0cd9ce454a26b8409ffc5c25fcf6` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:28.365915 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c9998b-5872-4d8c-b793-15e91df14ee1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=242c0cd9ce454a26b8409ffc5c25fcf6)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:28.367822 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: WARNING py.warnings [None req-06c9998b-5872-4d8c-b793-15e91df14ee1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 22:47:28.367822 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: warnings.warn(msg) Aug 10 22:47:28.367822 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: Aug 10 22:47:28.375209 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c9998b-5872-4d8c-b793-15e91df14ee1 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:28.400104 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70119e66-172f-489a-954c-cd3b7b8a7c9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:28.400509 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70119e66-172f-489a-954c-cd3b7b8a7c9e service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:28.400799 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70119e66-172f-489a-954c-cd3b7b8a7c9e service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:28.402174 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70119e66-172f-489a-954c-cd3b7b8a7c9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:28.434290 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70119e66-172f-489a-954c-cd3b7b8a7c9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:28.439042 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70119e66-172f-489a-954c-cd3b7b8a7c9e service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:28.467825 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 837/1664] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:28.517268 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-06c9998b-5872-4d8c-b793-15e91df14ee1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '242c0cd9ce454a26b8409ffc5c25fcf6', 'request_id': 'req-06c9998b-5872-4d8c-b793-15e91df14ee1'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:28.528924 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-06c9998b-5872-4d8c-b793-15e91df14ee1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '242c0cd9ce454a26b8409ffc5c25fcf6', 'request_id': 'req-06c9998b-5872-4d8c-b793-15e91df14ee1'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:28.539846 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-06c9998b-5872-4d8c-b793-15e91df14ee1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '242c0cd9ce454a26b8409ffc5c25fcf6', 'request_id': 'req-06c9998b-5872-4d8c-b793-15e91df14ee1'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:28.562224 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-06c9998b-5872-4d8c-b793-15e91df14ee1 admin admin] The token cache is being invalidate because project 242c0cd9ce454a26b8409ffc5c25fcf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:47:28.562509 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-06c9998b-5872-4d8c-b793-15e91df14ee1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:28.563503 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-06c9998b-5872-4d8c-b793-15e91df14ee1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:28.565747 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 828/1665] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:47:28 2022] DELETE /identity/v3/projects/242c0cd9ce454a26b8409ffc5c25fcf6 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:47:28.571104 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddbede-ecc4-4fc8-abef-90155e6768d8 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:28.571799 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddbede-ecc4-4fc8-abef-90155e6768d8 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:28.572242 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddbede-ecc4-4fc8-abef-90155e6768d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:28.603267 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-2fddbede-ecc4-4fc8-abef-90155e6768d8 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:28.604719 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-2fddbede-ecc4-4fc8-abef-90155e6768d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:28.706670 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 838/1666] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:28.712605 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36368078-bf1e-496b-8eb1-bcbea1d72dd4 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:28.713023 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36368078-bf1e-496b-8eb1-bcbea1d72dd4 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:28.713602 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36368078-bf1e-496b-8eb1-bcbea1d72dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:28.785049 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-36368078-bf1e-496b-8eb1-bcbea1d72dd4 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:28.786716 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-36368078-bf1e-496b-8eb1-bcbea1d72dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:28.809146 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 829/1667] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:28.816333 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fddbede-ecc4-4fc8-abef-90155e6768d8 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:28.829050 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:28.850014 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fddbede-ecc4-4fc8-abef-90155e6768d8 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:28.860315 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147dd713-5765-40d0-9492-a9a31b4a381e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:28.861158 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147dd713-5765-40d0-9492-a9a31b4a381e admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:28.861598 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147dd713-5765-40d0-9492-a9a31b4a381e admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:28.862029 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147dd713-5765-40d0-9492-a9a31b4a381e admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:28.863438 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147dd713-5765-40d0-9492-a9a31b4a381e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:47:28.864339 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147dd713-5765-40d0-9492-a9a31b4a381e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:28.868279 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147dd713-5765-40d0-9492-a9a31b4a381e admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:28.902597 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 839/1668] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:47:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:47:28.906481 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36368078-bf1e-496b-8eb1-bcbea1d72dd4 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:28.918730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36368078-bf1e-496b-8eb1-bcbea1d72dd4 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:28.933073 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1e470-2cbc-4393-b6a0-b570fc21d16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:28.934036 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1e470-2cbc-4393-b6a0-b570fc21d16d admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:28.934370 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1e470-2cbc-4393-b6a0-b570fc21d16d admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:28.934652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1e470-2cbc-4393-b6a0-b570fc21d16d admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:28.935393 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a1e470-2cbc-4393-b6a0-b570fc21d16d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:28.938632 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a1e470-2cbc-4393-b6a0-b570fc21d16d admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:29.041956 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 830/1669] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:47:28 2022] POST /identity/v3/projects => generated 367 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:29.046075 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147dd713-5765-40d0-9492-a9a31b4a381e admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:29.076738 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147dd713-5765-40d0-9492-a9a31b4a381e admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:29.262743 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb353312-748a-4e30-99f6-0e33419dac2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:29.264240 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb353312-748a-4e30-99f6-0e33419dac2d admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:29.265915 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb353312-748a-4e30-99f6-0e33419dac2d admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:29.266560 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb353312-748a-4e30-99f6-0e33419dac2d admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:29.274735 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb353312-748a-4e30-99f6-0e33419dac2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:29.279905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb353312-748a-4e30-99f6-0e33419dac2d admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:29.345522 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 840/1670] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:47:29 2022] POST /identity/v3/users => generated 284 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:29.347621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1e470-2cbc-4393-b6a0-b570fc21d16d admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:29.367205 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1e470-2cbc-4393-b6a0-b570fc21d16d admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:29.398737 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0328c714-0a5f-4cb7-81f5-67ddca1ed0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:29.399434 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328c714-0a5f-4cb7-81f5-67ddca1ed0a5 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:29.399719 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328c714-0a5f-4cb7-81f5-67ddca1ed0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:29.399996 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328c714-0a5f-4cb7-81f5-67ddca1ed0a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:29.400666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0328c714-0a5f-4cb7-81f5-67ddca1ed0a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:29.402591 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0328c714-0a5f-4cb7-81f5-67ddca1ed0a5 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:29.462738 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 831/1671] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:29 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:29.463798 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb353312-748a-4e30-99f6-0e33419dac2d admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:29.497689 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb353312-748a-4e30-99f6-0e33419dac2d admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:29.515979 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e4f28-ef65-416d-99d5-1fae764ad567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:29.516632 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e4f28-ef65-416d-99d5-1fae764ad567 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:29.516914 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e4f28-ef65-416d-99d5-1fae764ad567 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:29.518044 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e4f28-ef65-416d-99d5-1fae764ad567 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:29.518713 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e4f28-ef65-416d-99d5-1fae764ad567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:29.520379 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e4f28-ef65-416d-99d5-1fae764ad567 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:29.600156 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 841/1672] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:29 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:29.602319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0328c714-0a5f-4cb7-81f5-67ddca1ed0a5 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:29.624753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0328c714-0a5f-4cb7-81f5-67ddca1ed0a5 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:29.637972 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341974a7-2e58-43a3-84a5-8e1396a40618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:29.640823 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341974a7-2e58-43a3-84a5-8e1396a40618 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:29.641244 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341974a7-2e58-43a3-84a5-8e1396a40618 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:29.641909 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341974a7-2e58-43a3-84a5-8e1396a40618 admin admin] PATH_INFO: `/v3/projects/82f9668195434d03b0555d6a01938803/users/95345eb9d8bc460aa5ee56d2efe54c33/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:29.702229 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341974a7-2e58-43a3-84a5-8e1396a40618 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82f9668195434d03b0555d6a01938803, user_id=95345eb9d8bc460aa5ee56d2efe54c33, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:29.713322 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341974a7-2e58-43a3-84a5-8e1396a40618 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:29.736772 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 832/1673] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:47:29 2022] PUT /identity/v3/projects/82f9668195434d03b0555d6a01938803/users/95345eb9d8bc460aa5ee56d2efe54c33/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:29.753263 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f494d9-2995-4b98-b32e-c984c056e2dd None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:29.753939 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f494d9-2995-4b98-b32e-c984c056e2dd None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:29.754419 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f494d9-2995-4b98-b32e-c984c056e2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:29.854186 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-13f494d9-2995-4b98-b32e-c984c056e2dd None None] MFA Rules not processed for user `95345eb9d8bc460aa5ee56d2efe54c33`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:29.858131 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-13f494d9-2995-4b98-b32e-c984c056e2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:29.983471 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341974a7-2e58-43a3-84a5-8e1396a40618 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:29.986388 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-341974a7-2e58-43a3-84a5-8e1396a40618 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:30.157329 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:30.181642 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 842/1674] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 435 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:30.186079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f494d9-2995-4b98-b32e-c984c056e2dd None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:30.234128 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341974a7-2e58-43a3-84a5-8e1396a40618 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:30.243927 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0785fae5-1039-4332-b919-c366044731a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:30.244858 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785fae5-1039-4332-b919-c366044731a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:30.245328 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785fae5-1039-4332-b919-c366044731a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:30.245767 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785fae5-1039-4332-b919-c366044731a8 admin admin] PATH_INFO: `/v3/projects/09cce397513c41ceb2280e51b44b1a4d` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:30.247532 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0785fae5-1039-4332-b919-c366044731a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09cce397513c41ceb2280e51b44b1a4d)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:30.249753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0785fae5-1039-4332-b919-c366044731a8 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:30.331513 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:30.340246 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f494d9-2995-4b98-b32e-c984c056e2dd None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:30.364906 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-0785fae5-1039-4332-b919-c366044731a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09cce397513c41ceb2280e51b44b1a4d', 'request_id': 'req-0785fae5-1039-4332-b919-c366044731a8'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:30.385860 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-0785fae5-1039-4332-b919-c366044731a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09cce397513c41ceb2280e51b44b1a4d', 'request_id': 'req-0785fae5-1039-4332-b919-c366044731a8'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:30.395764 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-0785fae5-1039-4332-b919-c366044731a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09cce397513c41ceb2280e51b44b1a4d', 'request_id': 'req-0785fae5-1039-4332-b919-c366044731a8'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:30.435199 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-0785fae5-1039-4332-b919-c366044731a8 admin admin] The token cache is being invalidate because project 09cce397513c41ceb2280e51b44b1a4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:47:30.435691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-0785fae5-1039-4332-b919-c366044731a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:30.441753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-0785fae5-1039-4332-b919-c366044731a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:30.444347 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 833/1675] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:47:29 2022] DELETE /identity/v3/projects/09cce397513c41ceb2280e51b44b1a4d => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:47:30.448753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0785fae5-1039-4332-b919-c366044731a8 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:30.450426 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-0785fae5-1039-4332-b919-c366044731a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:30.493357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb015a-b76f-4e71-8f0f-3cee5efea95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:30.494030 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb015a-b76f-4e71-8f0f-3cee5efea95a admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:30.494304 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb015a-b76f-4e71-8f0f-3cee5efea95a admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:30.494584 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb015a-b76f-4e71-8f0f-3cee5efea95a admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:30.495228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb015a-b76f-4e71-8f0f-3cee5efea95a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:30.497184 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb015a-b76f-4e71-8f0f-3cee5efea95a admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:30.599020 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 843/1676] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:47:30 2022] POST /identity/v3/projects => generated 369 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:47:30.603368 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb015a-b76f-4e71-8f0f-3cee5efea95a admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:30.605220 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-cedb015a-b76f-4e71-8f0f-3cee5efea95a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:30.631819 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0785fae5-1039-4332-b919-c366044731a8 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:30.773541 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15b4ff-d498-4cfa-8d89-eed6868dd734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:30.774378 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15b4ff-d498-4cfa-8d89-eed6868dd734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:30.774843 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15b4ff-d498-4cfa-8d89-eed6868dd734 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:30.775243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15b4ff-d498-4cfa-8d89-eed6868dd734 admin admin] PATH_INFO: `/v3/users/81cfc1a8d5ab497fa4159ca6c8736cb7` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:30.778548 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15b4ff-d498-4cfa-8d89-eed6868dd734 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81cfc1a8d5ab497fa4159ca6c8736cb7)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:30.786998 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15b4ff-d498-4cfa-8d89-eed6868dd734 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:30.797504 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb015a-b76f-4e71-8f0f-3cee5efea95a admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:30.808298 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:30.811823 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57a0da-7e9c-47b5-9833-c593d6efc143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:30.812577 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57a0da-7e9c-47b5-9833-c593d6efc143 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:30.812863 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57a0da-7e9c-47b5-9833-c593d6efc143 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:30.813794 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57a0da-7e9c-47b5-9833-c593d6efc143 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:30.814855 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c57a0da-7e9c-47b5-9833-c593d6efc143 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:30.817091 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c57a0da-7e9c-47b5-9833-c593d6efc143 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:30.868519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 844/1677] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:47:30 2022] POST /identity/v3/users => generated 291 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:47:30.870126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57a0da-7e9c-47b5-9833-c593d6efc143 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:30.876435 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-8c57a0da-7e9c-47b5-9833-c593d6efc143 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:30.900669 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57a0da-7e9c-47b5-9833-c593d6efc143 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:30.902257 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-8a15b4ff-d498-4cfa-8d89-eed6868dd734 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81cfc1a8d5ab497fa4159ca6c8736cb7'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:30.918718 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6d0fb8-f8c2-4189-8f7f-b5b27d2c017f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:30.919386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d0fb8-f8c2-4189-8f7f-b5b27d2c017f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:30.919672 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d0fb8-f8c2-4189-8f7f-b5b27d2c017f admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:30.920084 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d0fb8-f8c2-4189-8f7f-b5b27d2c017f admin admin] PATH_INFO: `/v3/projects/b1f28560c6db49afa2cdfbf5dc583d1d` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:30.921078 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-8a15b4ff-d498-4cfa-8d89-eed6868dd734 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81cfc1a8d5ab497fa4159ca6c8736cb7'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:30.923554 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6d0fb8-f8c2-4189-8f7f-b5b27d2c017f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1f28560c6db49afa2cdfbf5dc583d1d)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:30.925484 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6d0fb8-f8c2-4189-8f7f-b5b27d2c017f admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:30.943121 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-8a15b4ff-d498-4cfa-8d89-eed6868dd734 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81cfc1a8d5ab497fa4159ca6c8736cb7'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:30.944009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-8a15b4ff-d498-4cfa-8d89-eed6868dd734 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81cfc1a8d5ab497fa4159ca6c8736cb7'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:30.948448 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-8a15b4ff-d498-4cfa-8d89-eed6868dd734 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81cfc1a8d5ab497fa4159ca6c8736cb7'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:30.973046 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 834/1678] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:47:30 2022] DELETE /identity/v3/users/81cfc1a8d5ab497fa4159ca6c8736cb7 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:30.973046 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15b4ff-d498-4cfa-8d89-eed6868dd734 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:30.973046 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-8a15b4ff-d498-4cfa-8d89-eed6868dd734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:31.050350 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-7a6d0fb8-f8c2-4189-8f7f-b5b27d2c017f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1f28560c6db49afa2cdfbf5dc583d1d', 'request_id': 'req-7a6d0fb8-f8c2-4189-8f7f-b5b27d2c017f'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:31.058595 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-7a6d0fb8-f8c2-4189-8f7f-b5b27d2c017f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1f28560c6db49afa2cdfbf5dc583d1d', 'request_id': 'req-7a6d0fb8-f8c2-4189-8f7f-b5b27d2c017f'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:31.080712 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-7a6d0fb8-f8c2-4189-8f7f-b5b27d2c017f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1f28560c6db49afa2cdfbf5dc583d1d', 'request_id': 'req-7a6d0fb8-f8c2-4189-8f7f-b5b27d2c017f'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:31.103141 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-7a6d0fb8-f8c2-4189-8f7f-b5b27d2c017f admin admin] The token cache is being invalidate because project b1f28560c6db49afa2cdfbf5dc583d1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:47:31.103711 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-7a6d0fb8-f8c2-4189-8f7f-b5b27d2c017f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:31.105014 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-7a6d0fb8-f8c2-4189-8f7f-b5b27d2c017f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:31.109999 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15b4ff-d498-4cfa-8d89-eed6868dd734 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:31.110849 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 845/1679] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:47:30 2022] DELETE /identity/v3/projects/b1f28560c6db49afa2cdfbf5dc583d1d => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:31.116030 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b0458-4eb7-4690-bfcc-d3a6111fa863 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:31.116316 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b0458-4eb7-4690-bfcc-d3a6111fa863 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:31.117021 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b0458-4eb7-4690-bfcc-d3a6111fa863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:31.136762 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:31.168574 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-c62b0458-4eb7-4690-bfcc-d3a6111fa863 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:31.169876 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-c62b0458-4eb7-4690-bfcc-d3a6111fa863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:31.255559 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b46a99-b255-4795-834f-e2c062791ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:31.265319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b46a99-b255-4795-834f-e2c062791ee5 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:31.265761 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b46a99-b255-4795-834f-e2c062791ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:31.266145 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b46a99-b255-4795-834f-e2c062791ee5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:31.266930 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b46a99-b255-4795-834f-e2c062791ee5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:31.268687 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b46a99-b255-4795-834f-e2c062791ee5 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:31.293366 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 835/1680] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:30 2022] GET /identity/v3/roles => generated 1420 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:31.298003 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b46a99-b255-4795-834f-e2c062791ee5 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:31.302818 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-25b46a99-b255-4795-834f-e2c062791ee5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:31.325639 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 846/1681] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:47:31.330607 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b46a99-b255-4795-834f-e2c062791ee5 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:31.333577 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a4836-0545-4c9c-94d0-bea24f22eca5 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:31.334373 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a4836-0545-4c9c-94d0-bea24f22eca5 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:31.334917 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a4836-0545-4c9c-94d0-bea24f22eca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:31.342920 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da795eeb-e043-4901-b736-b13ebf7d6d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:31.343754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da795eeb-e043-4901-b736-b13ebf7d6d26 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:31.344178 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da795eeb-e043-4901-b736-b13ebf7d6d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:31.344586 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da795eeb-e043-4901-b736-b13ebf7d6d26 admin admin] PATH_INFO: `/v3/projects/210742a89cc34af2bbc6eae9a3fd3a0e/users/0d540fb452954b268928ea0cdea302c9/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:31.384030 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-5c1a4836-0545-4c9c-94d0-bea24f22eca5 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:31.385188 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da795eeb-e043-4901-b736-b13ebf7d6d26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=210742a89cc34af2bbc6eae9a3fd3a0e, user_id=0d540fb452954b268928ea0cdea302c9, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:31.387272 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-5c1a4836-0545-4c9c-94d0-bea24f22eca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:31.388345 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da795eeb-e043-4901-b736-b13ebf7d6d26 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:31.411055 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 847/1682] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:31.414311 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 836/1683] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:47:31 2022] PUT /identity/v3/projects/210742a89cc34af2bbc6eae9a3fd3a0e/users/0d540fb452954b268928ea0cdea302c9/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:31.415759 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a4836-0545-4c9c-94d0-bea24f22eca5 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:31.416086 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da795eeb-e043-4901-b736-b13ebf7d6d26 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:31.434432 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a4836-0545-4c9c-94d0-bea24f22eca5 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:31.451314 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:31.626821 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da795eeb-e043-4901-b736-b13ebf7d6d26 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:31.627410 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75fadef-9a82-4ace-90c7-e9422a603502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:31.628241 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fadef-9a82-4ace-90c7-e9422a603502 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:31.631523 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fadef-9a82-4ace-90c7-e9422a603502 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:31.632075 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fadef-9a82-4ace-90c7-e9422a603502 admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:31.633841 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75fadef-9a82-4ace-90c7-e9422a603502 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:47:31.634450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75fadef-9a82-4ace-90c7-e9422a603502 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:31.636314 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75fadef-9a82-4ace-90c7-e9422a603502 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:31.646391 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73c3fc-fa59-4a58-bc53-3f4857c34188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:31.647324 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73c3fc-fa59-4a58-bc53-3f4857c34188 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:31.650252 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73c3fc-fa59-4a58-bc53-3f4857c34188 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:31.651186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73c3fc-fa59-4a58-bc53-3f4857c34188 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:31.654101 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a73c3fc-fa59-4a58-bc53-3f4857c34188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:31.659230 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a73c3fc-fa59-4a58-bc53-3f4857c34188 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:31.670101 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 848/1684] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:47:31.675410 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75fadef-9a82-4ace-90c7-e9422a603502 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:31.691008 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 837/1685] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:31 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:31.704523 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75fadef-9a82-4ace-90c7-e9422a603502 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:31.704836 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73c3fc-fa59-4a58-bc53-3f4857c34188 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:31.722384 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73c3fc-fa59-4a58-bc53-3f4857c34188 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:31.737756 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b2df3-d2c3-4b4e-a62c-0dd970486134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:31.738608 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b2df3-d2c3-4b4e-a62c-0dd970486134 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:31.739147 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b2df3-d2c3-4b4e-a62c-0dd970486134 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:31.739532 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b2df3-d2c3-4b4e-a62c-0dd970486134 admin admin] PATH_INFO: `/v3/projects/210742a89cc34af2bbc6eae9a3fd3a0e/users/0d540fb452954b268928ea0cdea302c9/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:31.746843 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b2df3-d2c3-4b4e-a62c-0dd970486134 admin admin] RBAC: Authorizing `identity:create_grant(project_id=210742a89cc34af2bbc6eae9a3fd3a0e, user_id=0d540fb452954b268928ea0cdea302c9, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:31.747377 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6a1c2-62f6-4908-8446-96f2b39811e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:31.748239 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6a1c2-62f6-4908-8446-96f2b39811e9 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:31.749555 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6a1c2-62f6-4908-8446-96f2b39811e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:31.750729 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b2df3-d2c3-4b4e-a62c-0dd970486134 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:31.751199 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6a1c2-62f6-4908-8446-96f2b39811e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:31.751997 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6a1c2-62f6-4908-8446-96f2b39811e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:31.756084 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6a1c2-62f6-4908-8446-96f2b39811e9 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:31.814193 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 838/1686] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:47:31 2022] PUT /identity/v3/projects/210742a89cc34af2bbc6eae9a3fd3a0e/users/0d540fb452954b268928ea0cdea302c9/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:31.816388 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eba078-27c0-437a-bb40-7e67d3b11995 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:31.819846 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eba078-27c0-437a-bb40-7e67d3b11995 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:31.820393 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eba078-27c0-437a-bb40-7e67d3b11995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:31.875481 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 849/1687] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:47:31 2022] POST /identity/v3/projects => generated 387 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:31.890437 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6a1c2-62f6-4908-8446-96f2b39811e9 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:31.930491 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-b7eba078-27c0-437a-bb40-7e67d3b11995 None None] MFA Rules not processed for user `0d540fb452954b268928ea0cdea302c9`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:31.934606 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-b7eba078-27c0-437a-bb40-7e67d3b11995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:32.091242 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6a1c2-62f6-4908-8446-96f2b39811e9 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:32.112859 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2326c287-bbb8-426a-a944-6ca6788bdf86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:32.113784 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326c287-bbb8-426a-a944-6ca6788bdf86 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:32.114212 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326c287-bbb8-426a-a944-6ca6788bdf86 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:32.114655 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326c287-bbb8-426a-a944-6ca6788bdf86 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:32.115437 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2326c287-bbb8-426a-a944-6ca6788bdf86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:32.118280 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2326c287-bbb8-426a-a944-6ca6788bdf86 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:32.167855 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:32.175556 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 850/1688] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:47:31 2022] POST /identity/v3/users => generated 294 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:32.181486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2326c287-bbb8-426a-a944-6ca6788bdf86 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:32.201640 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2326c287-bbb8-426a-a944-6ca6788bdf86 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:32.218672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 839/1689] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:31 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:32.219778 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eba078-27c0-437a-bb40-7e67d3b11995 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:32.227018 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00fff56-f717-4adc-a49c-bdea12bc3529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:32.227835 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00fff56-f717-4adc-a49c-bdea12bc3529 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:32.228117 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00fff56-f717-4adc-a49c-bdea12bc3529 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:32.228391 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00fff56-f717-4adc-a49c-bdea12bc3529 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:32.228485 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-b7eba078-27c0-437a-bb40-7e67d3b11995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:32.229056 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00fff56-f717-4adc-a49c-bdea12bc3529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:32.230737 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00fff56-f717-4adc-a49c-bdea12bc3529 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:32.263488 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 851/1690] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 22:47:32.267906 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00fff56-f717-4adc-a49c-bdea12bc3529 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:32.294398 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00fff56-f717-4adc-a49c-bdea12bc3529 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:32.310617 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889bd1fd-3788-4797-bba8-ad9a261db460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:32.311930 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bd1fd-3788-4797-bba8-ad9a261db460 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:32.312523 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bd1fd-3788-4797-bba8-ad9a261db460 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:32.313002 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bd1fd-3788-4797-bba8-ad9a261db460 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:32.313913 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889bd1fd-3788-4797-bba8-ad9a261db460 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:32.315759 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889bd1fd-3788-4797-bba8-ad9a261db460 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:32.363943 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eba078-27c0-437a-bb40-7e67d3b11995 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:32.365136 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 852/1691] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 22:47:32.367854 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889bd1fd-3788-4797-bba8-ad9a261db460 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:32.380397 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889bd1fd-3788-4797-bba8-ad9a261db460 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:32.381025 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe923f8-713a-47f0-87c6-162d7dc73905 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:32.381700 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe923f8-713a-47f0-87c6-162d7dc73905 service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:32.382064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe923f8-713a-47f0-87c6-162d7dc73905 service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:32.382551 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe923f8-713a-47f0-87c6-162d7dc73905 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:32.394014 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03756d9f-e145-4e63-9751-38426015a1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:32.394891 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03756d9f-e145-4e63-9751-38426015a1c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:32.395292 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03756d9f-e145-4e63-9751-38426015a1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:32.395987 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03756d9f-e145-4e63-9751-38426015a1c5 admin admin] PATH_INFO: `/v3/projects/b4995ba3721e40a785416a35d3c1872a/users/3a72673506f047edb03b69f6a3743a6e/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:32.432492 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe923f8-713a-47f0-87c6-162d7dc73905 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:32.434014 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe923f8-713a-47f0-87c6-162d7dc73905 service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:32.440549 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03756d9f-e145-4e63-9751-38426015a1c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4995ba3721e40a785416a35d3c1872a, user_id=3a72673506f047edb03b69f6a3743a6e, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:32.449012 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03756d9f-e145-4e63-9751-38426015a1c5 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:32.471479 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 840/1692] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:32 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:32.482513 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 853/1693] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:47:32 2022] PUT /identity/v3/projects/b4995ba3721e40a785416a35d3c1872a/users/3a72673506f047edb03b69f6a3743a6e/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:32.493686 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da6169-2ba6-478d-a6a3-602aa39b4edc None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:32.494022 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da6169-2ba6-478d-a6a3-602aa39b4edc None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:32.494305 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da6169-2ba6-478d-a6a3-602aa39b4edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:32.549098 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-e7da6169-2ba6-478d-a6a3-602aa39b4edc None None] MFA Rules not processed for user `3a72673506f047edb03b69f6a3743a6e`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:32.549470 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310041a8-4f55-4c95-b07c-96f3e6fa90cc None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:32.549836 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310041a8-4f55-4c95-b07c-96f3e6fa90cc None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:32.550193 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310041a8-4f55-4c95-b07c-96f3e6fa90cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:32.550491 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-e7da6169-2ba6-478d-a6a3-602aa39b4edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:32.574927 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:32.608037 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-310041a8-4f55-4c95-b07c-96f3e6fa90cc None None] MFA Rules not processed for user `12e6b0b71ecb4825aeab6735eb44e63e`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:32.609748 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-310041a8-4f55-4c95-b07c-96f3e6fa90cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:32.723947 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 854/1694] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:32 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:47:32.723947 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310041a8-4f55-4c95-b07c-96f3e6fa90cc None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:32.723947 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-310041a8-4f55-4c95-b07c-96f3e6fa90cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:32.728258 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 841/1695] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:32 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:47:32.732573 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da6169-2ba6-478d-a6a3-602aa39b4edc None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:32.800830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310041a8-4f55-4c95-b07c-96f3e6fa90cc None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:32.807078 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec3e56-cb5a-42a5-9913-bd61d038f3f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:32.809714 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec3e56-cb5a-42a5-9913-bd61d038f3f1 service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:32.810017 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec3e56-cb5a-42a5-9913-bd61d038f3f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:32.810301 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec3e56-cb5a-42a5-9913-bd61d038f3f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:32.812313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-9eec3e56-cb5a-42a5-9913-bd61d038f3f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:32.862895 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da6169-2ba6-478d-a6a3-602aa39b4edc None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:32.875598 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210d543-3bf2-44d0-bbd1-4075910da64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:32.876223 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210d543-3bf2-44d0-bbd1-4075910da64f admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:32.876500 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210d543-3bf2-44d0-bbd1-4075910da64f admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:32.876770 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210d543-3bf2-44d0-bbd1-4075910da64f admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:32.877430 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0210d543-3bf2-44d0-bbd1-4075910da64f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:32.879397 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0210d543-3bf2-44d0-bbd1-4075910da64f admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:32.891972 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:32.896047 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eec3e56-cb5a-42a5-9913-bd61d038f3f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:32.899139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eec3e56-cb5a-42a5-9913-bd61d038f3f1 service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:32.913321 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 855/1696] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:32.914540 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec3e56-cb5a-42a5-9913-bd61d038f3f1 service cinder] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:32.916316 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-9eec3e56-cb5a-42a5-9913-bd61d038f3f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:32.955572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 842/1697] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:47:32 2022] POST /identity/v3/projects => generated 387 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:47:32.964951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210d543-3bf2-44d0-bbd1-4075910da64f admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:33.018255 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec3e56-cb5a-42a5-9913-bd61d038f3f1 service cinder] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:33.104304 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210d543-3bf2-44d0-bbd1-4075910da64f admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:33.109774 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32650544-904e-4925-8fe2-1a59eebdcd60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:33.111113 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32650544-904e-4925-8fe2-1a59eebdcd60 service nova] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:33.111588 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32650544-904e-4925-8fe2-1a59eebdcd60 service nova] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:33.111933 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32650544-904e-4925-8fe2-1a59eebdcd60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:33.114075 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa990b-84b5-47a3-aaf5-92f6a7a6f640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:33.114643 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa990b-84b5-47a3-aaf5-92f6a7a6f640 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:33.114933 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa990b-84b5-47a3-aaf5-92f6a7a6f640 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:33.115618 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa990b-84b5-47a3-aaf5-92f6a7a6f640 admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:33.115819 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aa990b-84b5-47a3-aaf5-92f6a7a6f640 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:33.119144 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aa990b-84b5-47a3-aaf5-92f6a7a6f640 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:33.175749 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 843/1698] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:47:32 2022] POST /identity/v3/users => generated 300 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:47:33.177158 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa990b-84b5-47a3-aaf5-92f6a7a6f640 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:33.201030 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa990b-84b5-47a3-aaf5-92f6a7a6f640 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:33.216603 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32650544-904e-4925-8fe2-1a59eebdcd60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:33.218087 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c331b106-4253-4b12-9db9-82458855fd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:33.218487 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32650544-904e-4925-8fe2-1a59eebdcd60 service nova] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:33.218724 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c331b106-4253-4b12-9db9-82458855fd4b admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:33.219007 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c331b106-4253-4b12-9db9-82458855fd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:33.219281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c331b106-4253-4b12-9db9-82458855fd4b admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:33.219925 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c331b106-4253-4b12-9db9-82458855fd4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:33.221575 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c331b106-4253-4b12-9db9-82458855fd4b admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:33.242537 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 844/1699] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:33 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 22:47:33.243976 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c331b106-4253-4b12-9db9-82458855fd4b admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:33.250606 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 856/1700] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:32 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:33.259671 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c331b106-4253-4b12-9db9-82458855fd4b admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:33.265705 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d655254e-6828-4666-88a8-f1ceb3b2646a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:33.266462 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655254e-6828-4666-88a8-f1ceb3b2646a admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:33.266837 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655254e-6828-4666-88a8-f1ceb3b2646a admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:33.267176 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655254e-6828-4666-88a8-f1ceb3b2646a admin admin] PATH_INFO: `/v3/projects/19d883e7eb554993898667e94669b314/users/c9d52241808043cd85758a91a536913f/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:33.278818 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:33.295027 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d655254e-6828-4666-88a8-f1ceb3b2646a admin admin] RBAC: Authorizing `identity:create_grant(project_id=19d883e7eb554993898667e94669b314, user_id=c9d52241808043cd85758a91a536913f, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:33.300303 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d655254e-6828-4666-88a8-f1ceb3b2646a admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:33.323845 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 845/1701] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:47:33 2022] PUT /identity/v3/projects/19d883e7eb554993898667e94669b314/users/c9d52241808043cd85758a91a536913f/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:33.327693 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32650544-904e-4925-8fe2-1a59eebdcd60 service nova] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:33.344340 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32650544-904e-4925-8fe2-1a59eebdcd60 service nova] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:33.441827 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:33.508201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076e885-154c-41d0-a773-9f45e11900cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:33.508914 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076e885-154c-41d0-a773-9f45e11900cf admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:33.509327 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076e885-154c-41d0-a773-9f45e11900cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:33.509702 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076e885-154c-41d0-a773-9f45e11900cf admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:33.510286 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d076e885-154c-41d0-a773-9f45e11900cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:33.512028 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d076e885-154c-41d0-a773-9f45e11900cf admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:33.547602 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 857/1702] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:33 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:33.551929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d655254e-6828-4666-88a8-f1ceb3b2646a admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:33.583319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d655254e-6828-4666-88a8-f1ceb3b2646a admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:33.611259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b520590-658c-4d36-81ee-962b1b596ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:33.612533 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b520590-658c-4d36-81ee-962b1b596ded admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:33.613694 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b520590-658c-4d36-81ee-962b1b596ded admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:33.614231 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b520590-658c-4d36-81ee-962b1b596ded admin admin] PATH_INFO: `/v3/projects/19d883e7eb554993898667e94669b314/users/c9d52241808043cd85758a91a536913f/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:33.620781 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b520590-658c-4d36-81ee-962b1b596ded admin admin] RBAC: Authorizing `identity:create_grant(project_id=19d883e7eb554993898667e94669b314, user_id=c9d52241808043cd85758a91a536913f, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:33.626928 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b520590-658c-4d36-81ee-962b1b596ded admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:33.657514 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 846/1703] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:47:33 2022] PUT /identity/v3/projects/19d883e7eb554993898667e94669b314/users/c9d52241808043cd85758a91a536913f/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:33.669445 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09044be1-902b-4afa-9685-73961c65488c None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:33.669931 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09044be1-902b-4afa-9685-73961c65488c None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:33.670387 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09044be1-902b-4afa-9685-73961c65488c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:33.747192 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-09044be1-902b-4afa-9685-73961c65488c None None] MFA Rules not processed for user `c9d52241808043cd85758a91a536913f`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:33.748735 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-09044be1-902b-4afa-9685-73961c65488c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:33.976929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 858/1704] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:33 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:33.984526 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b520590-658c-4d36-81ee-962b1b596ded admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:34.184016 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b520590-658c-4d36-81ee-962b1b596ded admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:34.203213 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c79fdb-7706-4598-9894-0a38e995ec3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:34.203847 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c79fdb-7706-4598-9894-0a38e995ec3b service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:34.204126 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c79fdb-7706-4598-9894-0a38e995ec3b service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:34.204398 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c79fdb-7706-4598-9894-0a38e995ec3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:34.251724 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c79fdb-7706-4598-9894-0a38e995ec3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:34.253207 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c79fdb-7706-4598-9894-0a38e995ec3b service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:34.281260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 847/1705] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:33 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:34.458490 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09044be1-902b-4afa-9685-73961c65488c None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:34.699507 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09044be1-902b-4afa-9685-73961c65488c None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:34.713588 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb13bff-9bed-4218-84b9-9e13ed19dc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:34.713588 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb13bff-9bed-4218-84b9-9e13ed19dc42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:34.714146 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb13bff-9bed-4218-84b9-9e13ed19dc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:34.714426 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb13bff-9bed-4218-84b9-9e13ed19dc42 admin admin] PATH_INFO: `/v3/users/3a72673506f047edb03b69f6a3743a6e` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:34.718256 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb13bff-9bed-4218-84b9-9e13ed19dc42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a72673506f047edb03b69f6a3743a6e)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:34.720236 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb13bff-9bed-4218-84b9-9e13ed19dc42 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:34.823522 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-ceb13bff-9bed-4218-84b9-9e13ed19dc42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a72673506f047edb03b69f6a3743a6e'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:34.840939 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-ceb13bff-9bed-4218-84b9-9e13ed19dc42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a72673506f047edb03b69f6a3743a6e'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:34.842805 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:34.854574 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-ceb13bff-9bed-4218-84b9-9e13ed19dc42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a72673506f047edb03b69f6a3743a6e'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:34.857878 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-ceb13bff-9bed-4218-84b9-9e13ed19dc42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a72673506f047edb03b69f6a3743a6e'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:34.861776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-ceb13bff-9bed-4218-84b9-9e13ed19dc42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a72673506f047edb03b69f6a3743a6e'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:34.918035 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 859/1706] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:47:34 2022] DELETE /identity/v3/users/3a72673506f047edb03b69f6a3743a6e => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:34.920028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c5fcd-76c5-4963-aa66-d5fdbfc25ad0 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:34.920552 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c5fcd-76c5-4963-aa66-d5fdbfc25ad0 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:34.920849 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c5fcd-76c5-4963-aa66-d5fdbfc25ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:34.972806 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-959c5fcd-76c5-4963-aa66-d5fdbfc25ad0 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:34.977530 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-959c5fcd-76c5-4963-aa66-d5fdbfc25ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:35.140791 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb13bff-9bed-4218-84b9-9e13ed19dc42 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:35.143935 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-ceb13bff-9bed-4218-84b9-9e13ed19dc42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:35.152090 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 848/1707] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:35.154733 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c5fcd-76c5-4963-aa66-d5fdbfc25ad0 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:35.161511 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-959c5fcd-76c5-4963-aa66-d5fdbfc25ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:35.187584 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb13bff-9bed-4218-84b9-9e13ed19dc42 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:35.217980 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccad758-9453-4266-b2e2-9fb89d720aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:35.218654 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccad758-9453-4266-b2e2-9fb89d720aa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:35.218936 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccad758-9453-4266-b2e2-9fb89d720aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:35.219206 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccad758-9453-4266-b2e2-9fb89d720aa6 admin admin] PATH_INFO: `/v3/projects/b24b2bcb760f4e658ab6f2e7ff57847b` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:35.233332 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccad758-9453-4266-b2e2-9fb89d720aa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b24b2bcb760f4e658ab6f2e7ff57847b)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:35.250266 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccad758-9453-4266-b2e2-9fb89d720aa6 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:35.344594 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c5fcd-76c5-4963-aa66-d5fdbfc25ad0 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:35.393370 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-cccad758-9453-4266-b2e2-9fb89d720aa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b24b2bcb760f4e658ab6f2e7ff57847b', 'request_id': 'req-cccad758-9453-4266-b2e2-9fb89d720aa6'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:35.410898 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-cccad758-9453-4266-b2e2-9fb89d720aa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b24b2bcb760f4e658ab6f2e7ff57847b', 'request_id': 'req-cccad758-9453-4266-b2e2-9fb89d720aa6'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:35.438019 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:35.439439 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-cccad758-9453-4266-b2e2-9fb89d720aa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b24b2bcb760f4e658ab6f2e7ff57847b', 'request_id': 'req-cccad758-9453-4266-b2e2-9fb89d720aa6'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:35.468203 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-cccad758-9453-4266-b2e2-9fb89d720aa6 admin admin] The token cache is being invalidate because project b24b2bcb760f4e658ab6f2e7ff57847b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:47:35.468912 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-cccad758-9453-4266-b2e2-9fb89d720aa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:35.476987 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-cccad758-9453-4266-b2e2-9fb89d720aa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:35.481106 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 860/1708] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:47:35 2022] DELETE /identity/v3/projects/b24b2bcb760f4e658ab6f2e7ff57847b => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:47:35.491729 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474beb1-2ace-45e0-bc5a-e15e0fcfc504 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:35.492090 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474beb1-2ace-45e0-bc5a-e15e0fcfc504 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:35.492587 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474beb1-2ace-45e0-bc5a-e15e0fcfc504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:35.528245 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6d9b4-4255-4d99-80fc-7b1554d98f50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:35.528603 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6d9b4-4255-4d99-80fc-7b1554d98f50 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:35.528875 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6d9b4-4255-4d99-80fc-7b1554d98f50 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:35.529140 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6d9b4-4255-4d99-80fc-7b1554d98f50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:35.539641 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-cdf6d9b4-4255-4d99-80fc-7b1554d98f50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:35.572834 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-9474beb1-2ace-45e0-bc5a-e15e0fcfc504 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:35.576566 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-9474beb1-2ace-45e0-bc5a-e15e0fcfc504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:35.706319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf6d9b4-4255-4d99-80fc-7b1554d98f50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:35.707671 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf6d9b4-4255-4d99-80fc-7b1554d98f50 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:35.732780 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 861/1709] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:35.735123 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56e267-5473-48b5-b06e-78efa9a79cdd None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:35.735411 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56e267-5473-48b5-b06e-78efa9a79cdd None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:35.735689 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56e267-5473-48b5-b06e-78efa9a79cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:35.752422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 849/1710] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:35.792438 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-8d56e267-5473-48b5-b06e-78efa9a79cdd None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:35.795523 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-8d56e267-5473-48b5-b06e-78efa9a79cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:35.809267 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6d9b4-4255-4d99-80fc-7b1554d98f50 service neutron] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:35.810870 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-cdf6d9b4-4255-4d99-80fc-7b1554d98f50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:35.824716 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 862/1711] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:35.828840 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d56e267-5473-48b5-b06e-78efa9a79cdd None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:35.857536 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6d9b4-4255-4d99-80fc-7b1554d98f50 service neutron] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:35.869773 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d56e267-5473-48b5-b06e-78efa9a79cdd None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:35.874554 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b248b0-9557-4b1b-a0dd-c1d85c1132dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:35.875441 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b248b0-9557-4b1b-a0dd-c1d85c1132dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:35.875891 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b248b0-9557-4b1b-a0dd-c1d85c1132dd admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:35.876540 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b248b0-9557-4b1b-a0dd-c1d85c1132dd admin admin] PATH_INFO: `/v3/users/80a65ec30ace42ceb45e1a2d9d4c0b12` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:35.878028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b248b0-9557-4b1b-a0dd-c1d85c1132dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=80a65ec30ace42ceb45e1a2d9d4c0b12)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:35.881833 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b248b0-9557-4b1b-a0dd-c1d85c1132dd admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:35.886160 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88320511-b3d8-4b2d-86cd-baf0c47c8f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:35.887145 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88320511-b3d8-4b2d-86cd-baf0c47c8f31 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:35.887365 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88320511-b3d8-4b2d-86cd-baf0c47c8f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:35.887824 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88320511-b3d8-4b2d-86cd-baf0c47c8f31 admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:35.890345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88320511-b3d8-4b2d-86cd-baf0c47c8f31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:47:35.890765 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88320511-b3d8-4b2d-86cd-baf0c47c8f31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:35.892507 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88320511-b3d8-4b2d-86cd-baf0c47c8f31 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:35.930831 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 863/1712] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:47:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:47:35.931990 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88320511-b3d8-4b2d-86cd-baf0c47c8f31 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:35.948509 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88320511-b3d8-4b2d-86cd-baf0c47c8f31 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:35.961214 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c218ec-60af-4d0f-906d-66bb70d2e42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:35.962428 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c218ec-60af-4d0f-906d-66bb70d2e42b admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:35.963524 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c218ec-60af-4d0f-906d-66bb70d2e42b admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:35.964001 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c218ec-60af-4d0f-906d-66bb70d2e42b admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:35.964838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c218ec-60af-4d0f-906d-66bb70d2e42b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:35.971048 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c218ec-60af-4d0f-906d-66bb70d2e42b admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:35.998729 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-70b248b0-9557-4b1b-a0dd-c1d85c1132dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80a65ec30ace42ceb45e1a2d9d4c0b12'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:36.019608 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-70b248b0-9557-4b1b-a0dd-c1d85c1132dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80a65ec30ace42ceb45e1a2d9d4c0b12'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:36.050756 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-70b248b0-9557-4b1b-a0dd-c1d85c1132dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80a65ec30ace42ceb45e1a2d9d4c0b12'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:36.051717 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-70b248b0-9557-4b1b-a0dd-c1d85c1132dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80a65ec30ace42ceb45e1a2d9d4c0b12'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:36.052473 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-70b248b0-9557-4b1b-a0dd-c1d85c1132dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80a65ec30ace42ceb45e1a2d9d4c0b12'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:36.067611 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 864/1713] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:47:35 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:36.071159 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c218ec-60af-4d0f-906d-66bb70d2e42b admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:36.076676 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-c4c218ec-60af-4d0f-906d-66bb70d2e42b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:36.095186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 850/1714] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:47:35 2022] DELETE /identity/v3/users/80a65ec30ace42ceb45e1a2d9d4c0b12 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:47:36.100546 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579dcb3-972e-4c85-abb8-b4df4b9c5f52 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:36.101531 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579dcb3-972e-4c85-abb8-b4df4b9c5f52 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:36.101994 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579dcb3-972e-4c85-abb8-b4df4b9c5f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:36.121004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:36.161200 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-1579dcb3-972e-4c85-abb8-b4df4b9c5f52 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:36.162923 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-1579dcb3-972e-4c85-abb8-b4df4b9c5f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:36.302953 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:36.312864 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c218ec-60af-4d0f-906d-66bb70d2e42b admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:36.347921 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 851/1715] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:36.539424 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1579dcb3-972e-4c85-abb8-b4df4b9c5f52 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:36.541369 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-1579dcb3-972e-4c85-abb8-b4df4b9c5f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:36.555139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6c5a13-e979-4259-88d2-ceade7c25f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:36.560848 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c5a13-e979-4259-88d2-ceade7c25f46 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:36.561157 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c5a13-e979-4259-88d2-ceade7c25f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:36.563057 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c5a13-e979-4259-88d2-ceade7c25f46 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:36.567609 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6c5a13-e979-4259-88d2-ceade7c25f46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:36.571986 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6c5a13-e979-4259-88d2-ceade7c25f46 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:36.647256 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 865/1716] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:47:36 2022] POST /identity/v3/users => generated 293 bytes in 579 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:36.652031 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6c5a13-e979-4259-88d2-ceade7c25f46 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:36.715021 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6c5a13-e979-4259-88d2-ceade7c25f46 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:36.722159 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3d99f-7a90-435c-a2e1-0f93b34bd442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:36.723574 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3d99f-7a90-435c-a2e1-0f93b34bd442 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:36.724254 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3d99f-7a90-435c-a2e1-0f93b34bd442 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:36.724671 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3d99f-7a90-435c-a2e1-0f93b34bd442 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:36.726110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b3d99f-7a90-435c-a2e1-0f93b34bd442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:36.737255 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1579dcb3-972e-4c85-abb8-b4df4b9c5f52 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:36.744361 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b3b0b-4505-4689-9b69-5d3df9145fc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:36.745982 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b3b0b-4505-4689-9b69-5d3df9145fc0 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:36.746539 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b3b0b-4505-4689-9b69-5d3df9145fc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:36.746990 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b3b0b-4505-4689-9b69-5d3df9145fc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:36.759783 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2b3b0b-4505-4689-9b69-5d3df9145fc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:36.764870 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2b3b0b-4505-4689-9b69-5d3df9145fc0 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:36.775018 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b3d99f-7a90-435c-a2e1-0f93b34bd442 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:36.796715 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 852/1717] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 22:47:36.826255 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 866/1718] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:36 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:36.851994 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b3b0b-4505-4689-9b69-5d3df9145fc0 service neutron] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:36.871378 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b3b0b-4505-4689-9b69-5d3df9145fc0 service neutron] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:36.878356 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd07b18-94d5-415a-b64a-8dc1ff6c57e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:36.879499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd07b18-94d5-415a-b64a-8dc1ff6c57e0 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:36.880198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd07b18-94d5-415a-b64a-8dc1ff6c57e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:36.880641 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd07b18-94d5-415a-b64a-8dc1ff6c57e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:36.881392 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd07b18-94d5-415a-b64a-8dc1ff6c57e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:36.884623 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd07b18-94d5-415a-b64a-8dc1ff6c57e0 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:36.906738 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3d99f-7a90-435c-a2e1-0f93b34bd442 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:36.923283 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 853/1719] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:36 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:36.928024 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3d99f-7a90-435c-a2e1-0f93b34bd442 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:36.962730 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b743a0f5-4e73-443c-9dee-550bf2ae79ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:36.963474 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743a0f5-4e73-443c-9dee-550bf2ae79ab admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:36.963759 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743a0f5-4e73-443c-9dee-550bf2ae79ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:36.964034 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743a0f5-4e73-443c-9dee-550bf2ae79ab admin admin] PATH_INFO: `/v3/projects/cc86be049f1448e9a048c13f2d66a927/users/698492d7bf244d739b66dc5b1d29ec43/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:36.988094 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b743a0f5-4e73-443c-9dee-550bf2ae79ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc86be049f1448e9a048c13f2d66a927, user_id=698492d7bf244d739b66dc5b1d29ec43, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:36.990858 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b743a0f5-4e73-443c-9dee-550bf2ae79ab admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:37.033096 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 867/1720] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:47:36 2022] PUT /identity/v3/projects/cc86be049f1448e9a048c13f2d66a927/users/698492d7bf244d739b66dc5b1d29ec43/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:37.044360 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb703b3-edaa-4e5e-bb58-cd3b4a5666ad None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:37.045046 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb703b3-edaa-4e5e-bb58-cd3b4a5666ad None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:37.045598 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb703b3-edaa-4e5e-bb58-cd3b4a5666ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:37.093444 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-dbb703b3-edaa-4e5e-bb58-cd3b4a5666ad None None] MFA Rules not processed for user `698492d7bf244d739b66dc5b1d29ec43`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:37.094827 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-dbb703b3-edaa-4e5e-bb58-cd3b4a5666ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:37.273168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 854/1721] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:37.274955 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b743a0f5-4e73-443c-9dee-550bf2ae79ab admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:37.397789 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b743a0f5-4e73-443c-9dee-550bf2ae79ab admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:37.404365 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc65a2-f189-4014-8080-520e9f73a5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:37.405084 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc65a2-f189-4014-8080-520e9f73a5a4 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:37.405454 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc65a2-f189-4014-8080-520e9f73a5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:37.405957 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc65a2-f189-4014-8080-520e9f73a5a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:37.406656 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cc65a2-f189-4014-8080-520e9f73a5a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:37.409220 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cc65a2-f189-4014-8080-520e9f73a5a4 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:37.473492 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:37.494323 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 868/1722] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:47:37 2022] POST /identity/v3/projects => generated 387 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:37.497890 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb703b3-edaa-4e5e-bb58-cd3b4a5666ad None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:37.517353 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb703b3-edaa-4e5e-bb58-cd3b4a5666ad None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:37.705782 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5bedfb-b137-4b4d-bf1d-3f97ed24c5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:37.706548 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bedfb-b137-4b4d-bf1d-3f97ed24c5ae admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:37.706685 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bedfb-b137-4b4d-bf1d-3f97ed24c5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:37.706881 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bedfb-b137-4b4d-bf1d-3f97ed24c5ae admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:37.707515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5bedfb-b137-4b4d-bf1d-3f97ed24c5ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:37.709621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5bedfb-b137-4b4d-bf1d-3f97ed24c5ae admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:37.771336 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 855/1723] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:47:37 2022] POST /identity/v3/users => generated 300 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:37.776976 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc65a2-f189-4014-8080-520e9f73a5a4 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:37.802011 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc65a2-f189-4014-8080-520e9f73a5a4 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:37.812706 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ccea7-fee3-4b30-8cec-0a066262ab88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:37.814434 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ccea7-fee3-4b30-8cec-0a066262ab88 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:37.814858 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ccea7-fee3-4b30-8cec-0a066262ab88 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:37.815238 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ccea7-fee3-4b30-8cec-0a066262ab88 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:37.817224 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ccea7-fee3-4b30-8cec-0a066262ab88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:37.822967 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ccea7-fee3-4b30-8cec-0a066262ab88 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:37.864110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 869/1724] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:37 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:37.869746 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5bedfb-b137-4b4d-bf1d-3f97ed24c5ae admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:37.893788 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5bedfb-b137-4b4d-bf1d-3f97ed24c5ae admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:37.904686 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667b204-15c1-4b32-9a66-36593d7de2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:37.905970 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667b204-15c1-4b32-9a66-36593d7de2e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:37.906257 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667b204-15c1-4b32-9a66-36593d7de2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:37.906537 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667b204-15c1-4b32-9a66-36593d7de2e2 admin admin] PATH_INFO: `/v3/projects/c1191dfdda0641d58b6792dd8053c891/users/84e4c2bb6e4f4b828fd45ddcb7e6a66f/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:37.911461 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:37.960678 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3667b204-15c1-4b32-9a66-36593d7de2e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1191dfdda0641d58b6792dd8053c891, user_id=84e4c2bb6e4f4b828fd45ddcb7e6a66f, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:37.964332 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3667b204-15c1-4b32-9a66-36593d7de2e2 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:38.023639 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 856/1725] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:47:37 2022] PUT /identity/v3/projects/c1191dfdda0641d58b6792dd8053c891/users/84e4c2bb6e4f4b828fd45ddcb7e6a66f/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:38.027379 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ccea7-fee3-4b30-8cec-0a066262ab88 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:38.041268 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ccea7-fee3-4b30-8cec-0a066262ab88 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:38.166346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5945f62-9b56-487b-8686-635c8820901c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:38.168283 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5945f62-9b56-487b-8686-635c8820901c admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:38.169430 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5945f62-9b56-487b-8686-635c8820901c admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:38.169713 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5945f62-9b56-487b-8686-635c8820901c admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:38.173375 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5945f62-9b56-487b-8686-635c8820901c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:38.175210 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5945f62-9b56-487b-8686-635c8820901c admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:38.211991 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667b204-15c1-4b32-9a66-36593d7de2e2 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:38.212554 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 870/1726] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:38 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:38.239998 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667b204-15c1-4b32-9a66-36593d7de2e2 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:38.265191 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5945f62-9b56-487b-8686-635c8820901c admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:38.267601 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-c5945f62-9b56-487b-8686-635c8820901c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:38.278463 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b624d920-3d21-4a14-a4fe-730ce52bee16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:38.279448 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624d920-3d21-4a14-a4fe-730ce52bee16 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:38.279947 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624d920-3d21-4a14-a4fe-730ce52bee16 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:38.280628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624d920-3d21-4a14-a4fe-730ce52bee16 admin admin] PATH_INFO: `/v3/projects/c1191dfdda0641d58b6792dd8053c891/users/84e4c2bb6e4f4b828fd45ddcb7e6a66f/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:38.290941 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5945f62-9b56-487b-8686-635c8820901c admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:38.300311 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b624d920-3d21-4a14-a4fe-730ce52bee16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1191dfdda0641d58b6792dd8053c891, user_id=84e4c2bb6e4f4b828fd45ddcb7e6a66f, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:38.304862 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b624d920-3d21-4a14-a4fe-730ce52bee16 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:38.307751 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5069d8-768c-497a-a1da-97faa4bc70b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:38.308410 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5069d8-768c-497a-a1da-97faa4bc70b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:38.308688 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5069d8-768c-497a-a1da-97faa4bc70b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:38.311151 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5069d8-768c-497a-a1da-97faa4bc70b9 admin admin] PATH_INFO: `/v3/projects/b4995ba3721e40a785416a35d3c1872a` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:38.315140 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5069d8-768c-497a-a1da-97faa4bc70b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4995ba3721e40a785416a35d3c1872a)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:38.319090 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5069d8-768c-497a-a1da-97faa4bc70b9 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:38.335829 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 857/1727] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:47:38 2022] PUT /identity/v3/projects/c1191dfdda0641d58b6792dd8053c891/users/84e4c2bb6e4f4b828fd45ddcb7e6a66f/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:47:38.337365 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f4651-a420-44f0-9eec-67478d42c162 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:38.337815 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f4651-a420-44f0-9eec-67478d42c162 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:38.338249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f4651-a420-44f0-9eec-67478d42c162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:38.419406 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-181f4651-a420-44f0-9eec-67478d42c162 None None] MFA Rules not processed for user `12e6b0b71ecb4825aeab6735eb44e63e`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:38.420847 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-181f4651-a420-44f0-9eec-67478d42c162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:38.451709 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-2e5069d8-768c-497a-a1da-97faa4bc70b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4995ba3721e40a785416a35d3c1872a', 'request_id': 'req-2e5069d8-768c-497a-a1da-97faa4bc70b9'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:38.467510 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-2e5069d8-768c-497a-a1da-97faa4bc70b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4995ba3721e40a785416a35d3c1872a', 'request_id': 'req-2e5069d8-768c-497a-a1da-97faa4bc70b9'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:38.497545 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-2e5069d8-768c-497a-a1da-97faa4bc70b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4995ba3721e40a785416a35d3c1872a', 'request_id': 'req-2e5069d8-768c-497a-a1da-97faa4bc70b9'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:38.556644 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 858/1728] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:38 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:38.556644 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5421e-6185-4b35-8d13-4c59a000fc3a None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:38.556644 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5421e-6185-4b35-8d13-4c59a000fc3a None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:38.556644 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5421e-6185-4b35-8d13-4c59a000fc3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:38.595543 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-2e5069d8-768c-497a-a1da-97faa4bc70b9 admin admin] The token cache is being invalidate because project b4995ba3721e40a785416a35d3c1872a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:47:38.595543 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-2e5069d8-768c-497a-a1da-97faa4bc70b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:38.595543 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-2e5069d8-768c-497a-a1da-97faa4bc70b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:38.599847 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 871/1729] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:47:38 2022] DELETE /identity/v3/projects/b4995ba3721e40a785416a35d3c1872a => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:38.601751 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5069d8-768c-497a-a1da-97faa4bc70b9 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:38.604214 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-2e5069d8-768c-497a-a1da-97faa4bc70b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:38.644819 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-0db5421e-6185-4b35-8d13-4c59a000fc3a None None] MFA Rules not processed for user `84e4c2bb6e4f4b828fd45ddcb7e6a66f`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:38.648101 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-0db5421e-6185-4b35-8d13-4c59a000fc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:38.903409 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5069d8-768c-497a-a1da-97faa4bc70b9 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:38.915005 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823a962f-8956-4010-9ae1-def27f2fa4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:38.915826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a962f-8956-4010-9ae1-def27f2fa4f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:38.916114 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a962f-8956-4010-9ae1-def27f2fa4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:38.916526 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a962f-8956-4010-9ae1-def27f2fa4f1 admin admin] PATH_INFO: `/v3/users/c9d52241808043cd85758a91a536913f` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:38.920820 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823a962f-8956-4010-9ae1-def27f2fa4f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9d52241808043cd85758a91a536913f)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:38.923071 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823a962f-8956-4010-9ae1-def27f2fa4f1 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:38.973716 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:39.002401 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 859/1730] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:38 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 446 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:39.002401 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72695307-6bc1-44d5-8202-d98e37aef3a0 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:39.002401 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72695307-6bc1-44d5-8202-d98e37aef3a0 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:39.002401 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72695307-6bc1-44d5-8202-d98e37aef3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:39.079194 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-72695307-6bc1-44d5-8202-d98e37aef3a0 None None] MFA Rules not processed for user `0d4dc87fae144ab9a40e1f047330168d`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:39.080555 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-72695307-6bc1-44d5-8202-d98e37aef3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:39.099698 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-823a962f-8956-4010-9ae1-def27f2fa4f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9d52241808043cd85758a91a536913f'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:39.125879 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-823a962f-8956-4010-9ae1-def27f2fa4f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9d52241808043cd85758a91a536913f'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:39.146252 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-823a962f-8956-4010-9ae1-def27f2fa4f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9d52241808043cd85758a91a536913f'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:39.149816 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-823a962f-8956-4010-9ae1-def27f2fa4f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9d52241808043cd85758a91a536913f'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:39.150554 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-823a962f-8956-4010-9ae1-def27f2fa4f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9d52241808043cd85758a91a536913f'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:39.210345 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:39.241859 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 872/1731] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:47:38 2022] DELETE /identity/v3/users/c9d52241808043cd85758a91a536913f => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:39.248332 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823a962f-8956-4010-9ae1-def27f2fa4f1 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:39.254569 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-823a962f-8956-4010-9ae1-def27f2fa4f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:39.262742 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 860/1732] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:39.266471 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72695307-6bc1-44d5-8202-d98e37aef3a0 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:39.275708 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-72695307-6bc1-44d5-8202-d98e37aef3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:39.435247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823a962f-8956-4010-9ae1-def27f2fa4f1 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:39.447499 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18da88f-7fae-44e8-be49-bf33dcd1c319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:39.449568 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18da88f-7fae-44e8-be49-bf33dcd1c319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:39.450946 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18da88f-7fae-44e8-be49-bf33dcd1c319 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:39.451342 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18da88f-7fae-44e8-be49-bf33dcd1c319 admin admin] PATH_INFO: `/v3/users/95345eb9d8bc460aa5ee56d2efe54c33` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:39.453697 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18da88f-7fae-44e8-be49-bf33dcd1c319 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95345eb9d8bc460aa5ee56d2efe54c33)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:39.457418 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18da88f-7fae-44e8-be49-bf33dcd1c319 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:39.478818 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72695307-6bc1-44d5-8202-d98e37aef3a0 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:39.596601 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-b18da88f-7fae-44e8-be49-bf33dcd1c319 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95345eb9d8bc460aa5ee56d2efe54c33'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:39.608151 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-b18da88f-7fae-44e8-be49-bf33dcd1c319 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95345eb9d8bc460aa5ee56d2efe54c33'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:39.631472 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-b18da88f-7fae-44e8-be49-bf33dcd1c319 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95345eb9d8bc460aa5ee56d2efe54c33'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:39.637620 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-b18da88f-7fae-44e8-be49-bf33dcd1c319 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95345eb9d8bc460aa5ee56d2efe54c33'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:39.643086 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-b18da88f-7fae-44e8-be49-bf33dcd1c319 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95345eb9d8bc460aa5ee56d2efe54c33'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:39.686356 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 873/1733] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:47:39 2022] DELETE /identity/v3/users/95345eb9d8bc460aa5ee56d2efe54c33 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:39.687838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba3d84-9c10-48b2-a32c-93db2594e77b None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:39.688252 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba3d84-9c10-48b2-a32c-93db2594e77b None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:39.688697 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba3d84-9c10-48b2-a32c-93db2594e77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:39.716381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ad61b-36e5-4e21-b766-7dbffb7975ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:39.717054 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ad61b-36e5-4e21-b766-7dbffb7975ce service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:39.717430 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ad61b-36e5-4e21-b766-7dbffb7975ce service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:39.717783 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ad61b-36e5-4e21-b766-7dbffb7975ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:39.718403 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:39.724884 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-434ad61b-36e5-4e21-b766-7dbffb7975ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:39.750644 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-14ba3d84-9c10-48b2-a32c-93db2594e77b None None] MFA Rules not processed for user `0d4dc87fae144ab9a40e1f047330168d`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:39.751800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-14ba3d84-9c10-48b2-a32c-93db2594e77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:39.916889 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434ad61b-36e5-4e21-b766-7dbffb7975ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:39.920336 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434ad61b-36e5-4e21-b766-7dbffb7975ce service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:39.936061 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 874/1734] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:39.939295 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c25fb-08bd-471c-8c66-54c39ba7235c None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:39.939295 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c25fb-08bd-471c-8c66-54c39ba7235c None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:39.939486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c25fb-08bd-471c-8c66-54c39ba7235c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:39.968175 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 861/1735] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:39 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 701 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:39.971870 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5c82e-66f0-41ac-aee0-aed1e2a35490 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:39.972210 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5c82e-66f0-41ac-aee0-aed1e2a35490 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:39.972384 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5c82e-66f0-41ac-aee0-aed1e2a35490 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:39.975050 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 862/1736] 10.222.0.76 () {54 vars in 912 bytes} [Wed Aug 10 22:47:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:47:39.978708 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5c82e-66f0-41ac-aee0-aed1e2a35490 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:39.984601 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-3db5c82e-66f0-41ac-aee0-aed1e2a35490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:40.009786 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-320c25fb-08bd-471c-8c66-54c39ba7235c None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:40.013728 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-320c25fb-08bd-471c-8c66-54c39ba7235c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:40.142468 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:40.161052 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 875/1737] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:40.172776 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5c82e-66f0-41ac-aee0-aed1e2a35490 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:40.172975 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647341d5-3c5c-4917-a994-b6b58a8a4b71 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:40.173366 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647341d5-3c5c-4917-a994-b6b58a8a4b71 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:40.173685 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647341d5-3c5c-4917-a994-b6b58a8a4b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:40.183021 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c2ed25-ed04-412a-92d1-9ca446395d8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:40.183323 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2ed25-ed04-412a-92d1-9ca446395d8a service nova] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:40.183648 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2ed25-ed04-412a-92d1-9ca446395d8a service nova] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:40.184118 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2ed25-ed04-412a-92d1-9ca446395d8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:40.186403 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-79c2ed25-ed04-412a-92d1-9ca446395d8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:40.202478 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c2ed25-ed04-412a-92d1-9ca446395d8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:40.204462 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c2ed25-ed04-412a-92d1-9ca446395d8a service nova] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:40.220638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 863/1738] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:39 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:40.221571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c2ed25-ed04-412a-92d1-9ca446395d8a service nova] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:40.223107 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-79c2ed25-ed04-412a-92d1-9ca446395d8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:40.259104 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c2ed25-ed04-412a-92d1-9ca446395d8a service nova] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:40.262424 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-647341d5-3c5c-4917-a994-b6b58a8a4b71 None None] MFA Rules not processed for user `7fcb5a674ff94b3d8f79efbbe4959069`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:40.263720 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-647341d5-3c5c-4917-a994-b6b58a8a4b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:40.269864 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97682493-8337-43d4-9498-bff57c3c618f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:40.270716 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97682493-8337-43d4-9498-bff57c3c618f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:40.270993 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97682493-8337-43d4-9498-bff57c3c618f admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:40.271261 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97682493-8337-43d4-9498-bff57c3c618f admin admin] PATH_INFO: `/v3/users/698492d7bf244d739b66dc5b1d29ec43` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:40.274506 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97682493-8337-43d4-9498-bff57c3c618f admin admin] RBAC: Authorizing `identity:delete_user(user_id=698492d7bf244d739b66dc5b1d29ec43)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:40.281163 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97682493-8337-43d4-9498-bff57c3c618f admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:40.392822 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-97682493-8337-43d4-9498-bff57c3c618f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '698492d7bf244d739b66dc5b1d29ec43'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:40.395022 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 876/1739] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 10 22:47:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:40.396141 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647341d5-3c5c-4917-a994-b6b58a8a4b71 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:40.402332 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-647341d5-3c5c-4917-a994-b6b58a8a4b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:40.409202 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-97682493-8337-43d4-9498-bff57c3c618f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '698492d7bf244d739b66dc5b1d29ec43'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:40.431768 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647341d5-3c5c-4917-a994-b6b58a8a4b71 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:40.435938 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f6cf7-6cc4-4434-b3bb-b558e4336761 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:40.436600 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f6cf7-6cc4-4434-b3bb-b558e4336761 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:40.437045 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f6cf7-6cc4-4434-b3bb-b558e4336761 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:40.437413 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f6cf7-6cc4-4434-b3bb-b558e4336761 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:40.442198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-97682493-8337-43d4-9498-bff57c3c618f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '698492d7bf244d739b66dc5b1d29ec43'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:40.442929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-97682493-8337-43d4-9498-bff57c3c618f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '698492d7bf244d739b66dc5b1d29ec43'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:40.443759 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-97682493-8337-43d4-9498-bff57c3c618f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '698492d7bf244d739b66dc5b1d29ec43'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:40.472477 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 864/1740] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:47:40 2022] DELETE /identity/v3/users/698492d7bf244d739b66dc5b1d29ec43 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:40.472670 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7f6cf7-6cc4-4434-b3bb-b558e4336761 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:40.472670 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7f6cf7-6cc4-4434-b3bb-b558e4336761 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:40.475603 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c9edb-28ac-48fb-98ea-0682d4fe75fd None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:40.475897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c9edb-28ac-48fb-98ea-0682d4fe75fd None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:40.476176 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c9edb-28ac-48fb-98ea-0682d4fe75fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:40.489670 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 877/1741] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:40.514989 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f6cf7-6cc4-4434-b3bb-b558e4336761 service neutron] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:40.523809 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-c67c9edb-28ac-48fb-98ea-0682d4fe75fd None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:40.525057 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-c67c9edb-28ac-48fb-98ea-0682d4fe75fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:40.526425 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f6cf7-6cc4-4434-b3bb-b558e4336761 service neutron] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:40.575017 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:40.613983 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd320b2-3f9f-4603-959d-32a1fc5ec3c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:40.614533 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd320b2-3f9f-4603-959d-32a1fc5ec3c5 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:40.614879 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd320b2-3f9f-4603-959d-32a1fc5ec3c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:40.615228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd320b2-3f9f-4603-959d-32a1fc5ec3c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:40.621462 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-acd320b2-3f9f-4603-959d-32a1fc5ec3c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:40.641415 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd320b2-3f9f-4603-959d-32a1fc5ec3c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:40.643022 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd320b2-3f9f-4603-959d-32a1fc5ec3c5 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:40.653575 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 865/1743] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:40.653754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 878/1743] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 22:47:40.687774 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd320b2-3f9f-4603-959d-32a1fc5ec3c5 service neutron] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:40.689999 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-acd320b2-3f9f-4603-959d-32a1fc5ec3c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:40.705657 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd320b2-3f9f-4603-959d-32a1fc5ec3c5 service neutron] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:40.722279 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2559ad-dc29-4feb-86ff-4a67318b9a26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:40.723459 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2559ad-dc29-4feb-86ff-4a67318b9a26 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:40.723576 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2559ad-dc29-4feb-86ff-4a67318b9a26 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:40.723958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2559ad-dc29-4feb-86ff-4a67318b9a26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:40.743322 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2559ad-dc29-4feb-86ff-4a67318b9a26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:40.745166 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2559ad-dc29-4feb-86ff-4a67318b9a26 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:40.770199 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 879/1744] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:40.816968 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c9edb-28ac-48fb-98ea-0682d4fe75fd None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:40.822195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-c67c9edb-28ac-48fb-98ea-0682d4fe75fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:40.857228 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c9edb-28ac-48fb-98ea-0682d4fe75fd None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:40.869438 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80789620-d4df-4a82-99d8-3d8143ff726b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:40.870372 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80789620-d4df-4a82-99d8-3d8143ff726b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:40.870870 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80789620-d4df-4a82-99d8-3d8143ff726b admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:40.871367 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80789620-d4df-4a82-99d8-3d8143ff726b admin admin] PATH_INFO: `/v3/projects/dfa4e852819140a8ad723f54c756f607` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:40.874032 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80789620-d4df-4a82-99d8-3d8143ff726b admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfa4e852819140a8ad723f54c756f607)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:40.876724 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80789620-d4df-4a82-99d8-3d8143ff726b admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:40.969306 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:40.971515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-80789620-d4df-4a82-99d8-3d8143ff726b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfa4e852819140a8ad723f54c756f607', 'request_id': 'req-80789620-d4df-4a82-99d8-3d8143ff726b'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:40.988627 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-80789620-d4df-4a82-99d8-3d8143ff726b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfa4e852819140a8ad723f54c756f607', 'request_id': 'req-80789620-d4df-4a82-99d8-3d8143ff726b'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:41.003572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-80789620-d4df-4a82-99d8-3d8143ff726b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfa4e852819140a8ad723f54c756f607', 'request_id': 'req-80789620-d4df-4a82-99d8-3d8143ff726b'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:41.020968 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-80789620-d4df-4a82-99d8-3d8143ff726b admin admin] The token cache is being invalidate because project dfa4e852819140a8ad723f54c756f607 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:47:41.021093 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-80789620-d4df-4a82-99d8-3d8143ff726b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:41.022422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-80789620-d4df-4a82-99d8-3d8143ff726b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:41.024362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 866/1745] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:47:40 2022] DELETE /identity/v3/projects/dfa4e852819140a8ad723f54c756f607 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:41.028848 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa27a30-cd2a-4d31-a79e-c3b1e3a11e0f None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:41.029440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa27a30-cd2a-4d31-a79e-c3b1e3a11e0f None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:41.029440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa27a30-cd2a-4d31-a79e-c3b1e3a11e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:41.055428 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-afa27a30-cd2a-4d31-a79e-c3b1e3a11e0f None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:41.056755 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-afa27a30-cd2a-4d31-a79e-c3b1e3a11e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:41.062706 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7f4f6-cbd2-465d-85c1-2cebc093f677 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:41.062999 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7f4f6-cbd2-465d-85c1-2cebc093f677 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:41.063275 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7f4f6-cbd2-465d-85c1-2cebc093f677 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:41.064647 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 867/1746] 10.222.0.76 () {54 vars in 912 bytes} [Wed Aug 10 22:47:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 22:47:41.071551 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ad348-9a40-451c-a584-58b2c0c26938 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:41.071551 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ad348-9a40-451c-a584-58b2c0c26938 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:41.072210 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ad348-9a40-451c-a584-58b2c0c26938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:41.115538 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-520ad348-9a40-451c-a584-58b2c0c26938 None None] MFA Rules not processed for user `7fcb5a674ff94b3d8f79efbbe4959069`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:41.117995 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-520ad348-9a40-451c-a584-58b2c0c26938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:41.224495 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 880/1747] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:47:41.234139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef062b85-58d7-4a59-9aeb-09fad7e4a090 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:41.234566 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef062b85-58d7-4a59-9aeb-09fad7e4a090 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:41.234685 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef062b85-58d7-4a59-9aeb-09fad7e4a090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:41.246962 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 868/1748] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 10 22:47:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:47:41.277772 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-ef062b85-58d7-4a59-9aeb-09fad7e4a090 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:41.279020 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-ef062b85-58d7-4a59-9aeb-09fad7e4a090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:41.293338 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ad348-9a40-451c-a584-58b2c0c26938 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:41.294938 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-520ad348-9a40-451c-a584-58b2c0c26938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:41.296664 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 881/1749] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:47:41.300753 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef062b85-58d7-4a59-9aeb-09fad7e4a090 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:41.311155 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ad348-9a40-451c-a584-58b2c0c26938 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:41.322006 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef062b85-58d7-4a59-9aeb-09fad7e4a090 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:41.330795 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da69a1-4f40-44cc-bc61-0970fa313b88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:41.331039 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da69a1-4f40-44cc-bc61-0970fa313b88 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:41.331198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da69a1-4f40-44cc-bc61-0970fa313b88 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:41.331470 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da69a1-4f40-44cc-bc61-0970fa313b88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:41.333387 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093fd38a-6147-4547-9368-49796dd85a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:41.334013 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093fd38a-6147-4547-9368-49796dd85a9d admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:41.334292 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093fd38a-6147-4547-9368-49796dd85a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:41.334558 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093fd38a-6147-4547-9368-49796dd85a9d admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:41.335836 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093fd38a-6147-4547-9368-49796dd85a9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:47:41.336208 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093fd38a-6147-4547-9368-49796dd85a9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:41.338522 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093fd38a-6147-4547-9368-49796dd85a9d admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:41.355719 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86da69a1-4f40-44cc-bc61-0970fa313b88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:41.359777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86da69a1-4f40-44cc-bc61-0970fa313b88 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:41.371176 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 882/1750] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:47:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:47:41.375296 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a486604-1de4-4dd8-a633-c932f26c70cd None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:41.375588 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a486604-1de4-4dd8-a633-c932f26c70cd None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:41.375869 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a486604-1de4-4dd8-a633-c932f26c70cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:41.379102 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 869/1751] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 22:47:41.380008 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da69a1-4f40-44cc-bc61-0970fa313b88 service neutron] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:41.405427 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da69a1-4f40-44cc-bc61-0970fa313b88 service neutron] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:41.413685 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad51f3c-5a4d-48bc-a75b-27fa33e61900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:41.420758 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad51f3c-5a4d-48bc-a75b-27fa33e61900 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:41.421173 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad51f3c-5a4d-48bc-a75b-27fa33e61900 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:41.421610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad51f3c-5a4d-48bc-a75b-27fa33e61900 admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:41.422390 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad51f3c-5a4d-48bc-a75b-27fa33e61900 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:41.434606 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad51f3c-5a4d-48bc-a75b-27fa33e61900 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:41.439265 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-0a486604-1de4-4dd8-a633-c932f26c70cd None None] MFA Rules not processed for user `da22fc66a1d14fa983463331de110258`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:41.440494 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-0a486604-1de4-4dd8-a633-c932f26c70cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:41.505513 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 870/1752] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:47:41 2022] POST /identity/v3/projects => generated 405 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:41.507776 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad51f3c-5a4d-48bc-a75b-27fa33e61900 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:41.597115 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 883/1753] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:41.654518 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad51f3c-5a4d-48bc-a75b-27fa33e61900 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:41.728667 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70dd718-33e7-4314-b007-3e849ca0c535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:41.729420 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70dd718-33e7-4314-b007-3e849ca0c535 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:41.729717 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70dd718-33e7-4314-b007-3e849ca0c535 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:41.730000 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70dd718-33e7-4314-b007-3e849ca0c535 admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:41.738645 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70dd718-33e7-4314-b007-3e849ca0c535 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:41.742909 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70dd718-33e7-4314-b007-3e849ca0c535 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:41.797905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a486604-1de4-4dd8-a633-c932f26c70cd None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:41.798419 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 871/1754] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:47:41 2022] POST /identity/v3/users => generated 303 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:47:41.815587 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a486604-1de4-4dd8-a633-c932f26c70cd None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:41.826555 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52826f48-da2d-4180-ba9b-fd7270488b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:41.827272 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52826f48-da2d-4180-ba9b-fd7270488b63 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:41.827584 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52826f48-da2d-4180-ba9b-fd7270488b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:41.827837 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52826f48-da2d-4180-ba9b-fd7270488b63 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:41.828497 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52826f48-da2d-4180-ba9b-fd7270488b63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:41.830172 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52826f48-da2d-4180-ba9b-fd7270488b63 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:41.853343 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 884/1755] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:41 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:41.857247 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70dd718-33e7-4314-b007-3e849ca0c535 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:41.883020 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70dd718-33e7-4314-b007-3e849ca0c535 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:41.896338 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c529be-9fe3-44e3-997d-1ad46caebe0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:41.897047 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c529be-9fe3-44e3-997d-1ad46caebe0a admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:41.897404 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c529be-9fe3-44e3-997d-1ad46caebe0a admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:41.897730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c529be-9fe3-44e3-997d-1ad46caebe0a admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:41.898782 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c529be-9fe3-44e3-997d-1ad46caebe0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:41.900465 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c529be-9fe3-44e3-997d-1ad46caebe0a admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:41.910911 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52826f48-da2d-4180-ba9b-fd7270488b63 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:41.912913 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-52826f48-da2d-4180-ba9b-fd7270488b63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:41.938954 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52826f48-da2d-4180-ba9b-fd7270488b63 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:41.942952 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 872/1756] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:41 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:41.962610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c529be-9fe3-44e3-997d-1ad46caebe0a admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:41.971604 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2f438-2c2a-4912-b8d0-d1be602986c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:41.972305 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2f438-2c2a-4912-b8d0-d1be602986c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:41.972538 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2f438-2c2a-4912-b8d0-d1be602986c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:41.972815 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2f438-2c2a-4912-b8d0-d1be602986c9 admin admin] PATH_INFO: `/v3/projects/19d883e7eb554993898667e94669b314` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:41.974374 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2f438-2c2a-4912-b8d0-d1be602986c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19d883e7eb554993898667e94669b314)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:41.976918 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2f438-2c2a-4912-b8d0-d1be602986c9 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:42.006187 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c529be-9fe3-44e3-997d-1ad46caebe0a admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:42.020555 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34e4b0-3426-4bc3-a258-2e205c5d018a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:42.023053 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34e4b0-3426-4bc3-a258-2e205c5d018a admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:42.023434 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34e4b0-3426-4bc3-a258-2e205c5d018a admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:42.023912 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34e4b0-3426-4bc3-a258-2e205c5d018a admin admin] PATH_INFO: `/v3/projects/a86160e5bd874af4b18cb5f2dbccf4b0/users/b874dd1d3675426ea36bc55e7c660c85/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:42.063375 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:42.086240 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f34e4b0-3426-4bc3-a258-2e205c5d018a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a86160e5bd874af4b18cb5f2dbccf4b0, user_id=b874dd1d3675426ea36bc55e7c660c85, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:42.102241 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f34e4b0-3426-4bc3-a258-2e205c5d018a admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:42.134393 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 873/1757] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:47:41 2022] PUT /identity/v3/projects/a86160e5bd874af4b18cb5f2dbccf4b0/users/b874dd1d3675426ea36bc55e7c660c85/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:42.135932 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34e4b0-3426-4bc3-a258-2e205c5d018a admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:42.137808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-9f34e4b0-3426-4bc3-a258-2e205c5d018a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:42.146319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:DELETE=3 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:42.156439 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-3ff2f438-2c2a-4912-b8d0-d1be602986c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19d883e7eb554993898667e94669b314', 'request_id': 'req-3ff2f438-2c2a-4912-b8d0-d1be602986c9'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:42.175265 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-3ff2f438-2c2a-4912-b8d0-d1be602986c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19d883e7eb554993898667e94669b314', 'request_id': 'req-3ff2f438-2c2a-4912-b8d0-d1be602986c9'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:42.187739 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-3ff2f438-2c2a-4912-b8d0-d1be602986c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19d883e7eb554993898667e94669b314', 'request_id': 'req-3ff2f438-2c2a-4912-b8d0-d1be602986c9'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:42.208686 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-3ff2f438-2c2a-4912-b8d0-d1be602986c9 admin admin] The token cache is being invalidate because project 19d883e7eb554993898667e94669b314 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:47:42.209122 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-3ff2f438-2c2a-4912-b8d0-d1be602986c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:42.210996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-3ff2f438-2c2a-4912-b8d0-d1be602986c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:42.213205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 885/1758] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:47:41 2022] DELETE /identity/v3/projects/19d883e7eb554993898667e94669b314 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:42.243966 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2f438-2c2a-4912-b8d0-d1be602986c9 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:42.247640 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-3ff2f438-2c2a-4912-b8d0-d1be602986c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:42.265386 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34e4b0-3426-4bc3-a258-2e205c5d018a admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:42.270851 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3aaa71-871d-44c2-b4f5-86c00420fc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:42.271199 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3aaa71-871d-44c2-b4f5-86c00420fc87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:42.271479 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3aaa71-871d-44c2-b4f5-86c00420fc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:42.272138 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3aaa71-871d-44c2-b4f5-86c00420fc87 admin admin] PATH_INFO: `/v3/projects/82f9668195434d03b0555d6a01938803` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:42.274706 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3aaa71-871d-44c2-b4f5-86c00420fc87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82f9668195434d03b0555d6a01938803)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:42.278677 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3aaa71-871d-44c2-b4f5-86c00420fc87 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:42.360210 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-8d3aaa71-871d-44c2-b4f5-86c00420fc87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82f9668195434d03b0555d6a01938803', 'request_id': 'req-8d3aaa71-871d-44c2-b4f5-86c00420fc87'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:42.378630 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-8d3aaa71-871d-44c2-b4f5-86c00420fc87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82f9668195434d03b0555d6a01938803', 'request_id': 'req-8d3aaa71-871d-44c2-b4f5-86c00420fc87'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:42.389681 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-8d3aaa71-871d-44c2-b4f5-86c00420fc87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82f9668195434d03b0555d6a01938803', 'request_id': 'req-8d3aaa71-871d-44c2-b4f5-86c00420fc87'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:42.403446 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-8d3aaa71-871d-44c2-b4f5-86c00420fc87 admin admin] The token cache is being invalidate because project 82f9668195434d03b0555d6a01938803 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:47:42.413777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-8d3aaa71-871d-44c2-b4f5-86c00420fc87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:42.413777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-8d3aaa71-871d-44c2-b4f5-86c00420fc87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:42.413777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 874/1759] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:47:42 2022] DELETE /identity/v3/projects/82f9668195434d03b0555d6a01938803 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:42.413777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197ddbd-d53b-49cb-aa0d-480b74bac5dc None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:42.413777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197ddbd-d53b-49cb-aa0d-480b74bac5dc None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:42.413777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197ddbd-d53b-49cb-aa0d-480b74bac5dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:42.433660 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2f438-2c2a-4912-b8d0-d1be602986c9 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:42.448429 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-3197ddbd-d53b-49cb-aa0d-480b74bac5dc None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:42.449703 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-3197ddbd-d53b-49cb-aa0d-480b74bac5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:42.566650 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 875/1760] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:42.568162 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0892be-4015-4b0b-b217-df6fcf8ea51e None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:42.568455 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0892be-4015-4b0b-b217-df6fcf8ea51e None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:42.568728 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0892be-4015-4b0b-b217-df6fcf8ea51e None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:42.572231 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 876/1761] 10.222.0.76 () {54 vars in 961 bytes} [Wed Aug 10 22:47:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:47:42.575501 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9742e02-99fa-4fe0-a64f-953d8290f4c2 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:42.575787 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9742e02-99fa-4fe0-a64f-953d8290f4c2 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:42.579095 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9742e02-99fa-4fe0-a64f-953d8290f4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:42.607665 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449902d-25c1-4150-bfff-2a339238f041 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:42.609226 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449902d-25c1-4150-bfff-2a339238f041 service nova] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:42.609684 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449902d-25c1-4150-bfff-2a339238f041 service nova] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:42.610113 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449902d-25c1-4150-bfff-2a339238f041 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:42.612533 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-b9742e02-99fa-4fe0-a64f-953d8290f4c2 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:42.612784 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-b449902d-25c1-4150-bfff-2a339238f041 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:42.613862 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-b9742e02-99fa-4fe0-a64f-953d8290f4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:42.631640 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 877/1762] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:42.632870 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9742e02-99fa-4fe0-a64f-953d8290f4c2 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:42.639143 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-b9742e02-99fa-4fe0-a64f-953d8290f4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:42.672165 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9742e02-99fa-4fe0-a64f-953d8290f4c2 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:42.685330 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7355399-a5c2-4aad-bedb-1a097fffeb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:42.690216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7355399-a5c2-4aad-bedb-1a097fffeb3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:42.690528 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7355399-a5c2-4aad-bedb-1a097fffeb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:42.690829 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7355399-a5c2-4aad-bedb-1a097fffeb3b admin admin] PATH_INFO: `/v3/projects/cc86be049f1448e9a048c13f2d66a927` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:42.693438 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7355399-a5c2-4aad-bedb-1a097fffeb3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc86be049f1448e9a048c13f2d66a927)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:42.702205 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7355399-a5c2-4aad-bedb-1a097fffeb3b admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:42.714521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:42.750231 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b449902d-25c1-4150-bfff-2a339238f041 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:42.753937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b449902d-25c1-4150-bfff-2a339238f041 service nova] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:42.768727 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 886/1763] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:42.771459 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449902d-25c1-4150-bfff-2a339238f041 service nova] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:42.774622 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-b449902d-25c1-4150-bfff-2a339238f041 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:42.794111 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:42.798138 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449902d-25c1-4150-bfff-2a339238f041 service nova] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:42.808544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a64984-1515-408c-9e2a-8be9be4093c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:42.809805 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a64984-1515-408c-9e2a-8be9be4093c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:42.810466 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a64984-1515-408c-9e2a-8be9be4093c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:42.810985 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a64984-1515-408c-9e2a-8be9be4093c6 admin admin] PATH_INFO: `/v3/users/0d540fb452954b268928ea0cdea302c9` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:42.813816 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a64984-1515-408c-9e2a-8be9be4093c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d540fb452954b268928ea0cdea302c9)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:42.820215 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a64984-1515-408c-9e2a-8be9be4093c6 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:42.868701 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e7355399-a5c2-4aad-bedb-1a097fffeb3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc86be049f1448e9a048c13f2d66a927', 'request_id': 'req-e7355399-a5c2-4aad-bedb-1a097fffeb3b'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:42.897880 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e7355399-a5c2-4aad-bedb-1a097fffeb3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc86be049f1448e9a048c13f2d66a927', 'request_id': 'req-e7355399-a5c2-4aad-bedb-1a097fffeb3b'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:42.912201 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e7355399-a5c2-4aad-bedb-1a097fffeb3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc86be049f1448e9a048c13f2d66a927', 'request_id': 'req-e7355399-a5c2-4aad-bedb-1a097fffeb3b'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:42.929522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e7355399-a5c2-4aad-bedb-1a097fffeb3b admin admin] The token cache is being invalidate because project cc86be049f1448e9a048c13f2d66a927 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:47:42.930132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e7355399-a5c2-4aad-bedb-1a097fffeb3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:42.937813 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-91a64984-1515-408c-9e2a-8be9be4093c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d540fb452954b268928ea0cdea302c9'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:42.941930 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e7355399-a5c2-4aad-bedb-1a097fffeb3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:42.948398 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 878/1764] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:47:42 2022] DELETE /identity/v3/projects/cc86be049f1448e9a048c13f2d66a927 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:42.949986 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608673d-e300-49bc-bc44-a21cf281382e None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:42.950310 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608673d-e300-49bc-bc44-a21cf281382e None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:42.950591 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608673d-e300-49bc-bc44-a21cf281382e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:42.958538 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-91a64984-1515-408c-9e2a-8be9be4093c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d540fb452954b268928ea0cdea302c9'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:42.973821 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-91a64984-1515-408c-9e2a-8be9be4093c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d540fb452954b268928ea0cdea302c9'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:42.974830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-91a64984-1515-408c-9e2a-8be9be4093c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d540fb452954b268928ea0cdea302c9'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:42.976167 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-91a64984-1515-408c-9e2a-8be9be4093c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d540fb452954b268928ea0cdea302c9'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:43.009692 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-0608673d-e300-49bc-bc44-a21cf281382e None None] MFA Rules not processed for user `405738ed219b4fdc8e3c07706a13e5a2`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:43.013480 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-0608673d-e300-49bc-bc44-a21cf281382e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:43.024165 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 887/1765] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:47:42 2022] DELETE /identity/v3/users/0d540fb452954b268928ea0cdea302c9 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:43.028311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a64984-1515-408c-9e2a-8be9be4093c6 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:43.029956 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-91a64984-1515-408c-9e2a-8be9be4093c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:43.158508 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a64984-1515-408c-9e2a-8be9be4093c6 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:43.173216 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11548a5-dfc3-4c9c-9dcf-fda545ceaaa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:43.175622 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11548a5-dfc3-4c9c-9dcf-fda545ceaaa2 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:43.176309 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11548a5-dfc3-4c9c-9dcf-fda545ceaaa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:43.177475 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11548a5-dfc3-4c9c-9dcf-fda545ceaaa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:43.184824 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-a11548a5-dfc3-4c9c-9dcf-fda545ceaaa2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:43.190824 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 879/1766] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:43.192362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f3bee-106c-4f73-b334-acfb46eef872 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:43.192695 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f3bee-106c-4f73-b334-acfb46eef872 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:43.192970 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f3bee-106c-4f73-b334-acfb46eef872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:43.246386 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-3b2f3bee-106c-4f73-b334-acfb46eef872 None None] MFA Rules not processed for user `9f46dee0f73e49eab54c11556e3b86bf`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:43.247804 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-3b2f3bee-106c-4f73-b334-acfb46eef872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:43.359452 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11548a5-dfc3-4c9c-9dcf-fda545ceaaa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:43.361382 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11548a5-dfc3-4c9c-9dcf-fda545ceaaa2 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:43.388363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 888/1767] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:43.388579 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 880/1768] 10.222.0.76 () {58 vars in 1058 bytes} [Wed Aug 10 22:47:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:43.389960 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fb5f8-78eb-4604-ae1a-0c6b98c805af None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:43.390235 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fb5f8-78eb-4604-ae1a-0c6b98c805af None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:43.390341 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11548a5-dfc3-4c9c-9dcf-fda545ceaaa2 service neutron] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:43.390562 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fb5f8-78eb-4604-ae1a-0c6b98c805af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:43.392591 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-a11548a5-dfc3-4c9c-9dcf-fda545ceaaa2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:43.405502 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:43.410267 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11548a5-dfc3-4c9c-9dcf-fda545ceaaa2 service neutron] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:43.421509 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fafd7e-c360-48bf-aef6-3c438aaf3557 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f46dee0f73e49eab54c11556e3b86bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:43.423041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fafd7e-c360-48bf-aef6-3c438aaf3557 service placement] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:43.423357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fafd7e-c360-48bf-aef6-3c438aaf3557 service placement] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:43.423635 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fafd7e-c360-48bf-aef6-3c438aaf3557 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:43.426346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-79fafd7e-c360-48bf-aef6-3c438aaf3557 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:43.438275 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-757fb5f8-78eb-4604-ae1a-0c6b98c805af None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:43.439762 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-757fb5f8-78eb-4604-ae1a-0c6b98c805af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:43.456869 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fafd7e-c360-48bf-aef6-3c438aaf3557 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:43.459229 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fafd7e-c360-48bf-aef6-3c438aaf3557 service placement] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:43.464575 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 881/1769] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:43.468436 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757fb5f8-78eb-4604-ae1a-0c6b98c805af None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:43.478602 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-757fb5f8-78eb-4604-ae1a-0c6b98c805af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:43.480469 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 889/1770] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:43.481450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fafd7e-c360-48bf-aef6-3c438aaf3557 service placement] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:43.482900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-79fafd7e-c360-48bf-aef6-3c438aaf3557 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:43.520105 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757fb5f8-78eb-4604-ae1a-0c6b98c805af None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:43.529589 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a897dc-5809-4e4a-9262-fa9c270e6f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:43.530344 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a897dc-5809-4e4a-9262-fa9c270e6f4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:43.530688 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a897dc-5809-4e4a-9262-fa9c270e6f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:43.531439 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a897dc-5809-4e4a-9262-fa9c270e6f4c admin admin] PATH_INFO: `/v3/users/84e4c2bb6e4f4b828fd45ddcb7e6a66f` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:43.532324 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a897dc-5809-4e4a-9262-fa9c270e6f4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=84e4c2bb6e4f4b828fd45ddcb7e6a66f)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:43.541699 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a897dc-5809-4e4a-9262-fa9c270e6f4c admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:43.620222 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:43.681800 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f1a897dc-5809-4e4a-9262-fa9c270e6f4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84e4c2bb6e4f4b828fd45ddcb7e6a66f'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:43.697897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f1a897dc-5809-4e4a-9262-fa9c270e6f4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84e4c2bb6e4f4b828fd45ddcb7e6a66f'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:43.718661 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f1a897dc-5809-4e4a-9262-fa9c270e6f4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84e4c2bb6e4f4b828fd45ddcb7e6a66f'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:43.719978 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f1a897dc-5809-4e4a-9262-fa9c270e6f4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84e4c2bb6e4f4b828fd45ddcb7e6a66f'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:43.721025 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f1a897dc-5809-4e4a-9262-fa9c270e6f4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84e4c2bb6e4f4b828fd45ddcb7e6a66f'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:43.732525 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fafd7e-c360-48bf-aef6-3c438aaf3557 service placement] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:43.750868 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 882/1771] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:47:43 2022] DELETE /identity/v3/users/84e4c2bb6e4f4b828fd45ddcb7e6a66f => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:43.753418 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca89c3-26bd-4187-b840-f8c15d49a916 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:43.753983 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a897dc-5809-4e4a-9262-fa9c270e6f4c admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:43.756540 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca89c3-26bd-4187-b840-f8c15d49a916 service nova] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:43.758434 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-f1a897dc-5809-4e4a-9262-fa9c270e6f4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:43.761474 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca89c3-26bd-4187-b840-f8c15d49a916 service nova] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:43.761795 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca89c3-26bd-4187-b840-f8c15d49a916 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:43.767485 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-30ca89c3-26bd-4187-b840-f8c15d49a916 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:43.806137 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ca89c3-26bd-4187-b840-f8c15d49a916 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:43.808655 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ca89c3-26bd-4187-b840-f8c15d49a916 service nova] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:43.846097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 890/1772] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:43.846998 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca89c3-26bd-4187-b840-f8c15d49a916 service nova] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:43.853670 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-30ca89c3-26bd-4187-b840-f8c15d49a916 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:43.890250 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca89c3-26bd-4187-b840-f8c15d49a916 service nova] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:43.986752 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a897dc-5809-4e4a-9262-fa9c270e6f4c admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:43.994409 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52861ffc-6354-4c77-9623-4f96b41cfbef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f46dee0f73e49eab54c11556e3b86bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:43.995110 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52861ffc-6354-4c77-9623-4f96b41cfbef service placement] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:43.995396 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52861ffc-6354-4c77-9623-4f96b41cfbef service placement] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:43.995669 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52861ffc-6354-4c77-9623-4f96b41cfbef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:43.998139 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-52861ffc-6354-4c77-9623-4f96b41cfbef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:44.014806 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52861ffc-6354-4c77-9623-4f96b41cfbef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:44.016442 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52861ffc-6354-4c77-9623-4f96b41cfbef service placement] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:44.031078 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 883/1773] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:44.186148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cddf54-c22f-4f9e-a9fc-b107b0555315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:44.187574 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cddf54-c22f-4f9e-a9fc-b107b0555315 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:44.190116 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cddf54-c22f-4f9e-a9fc-b107b0555315 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:44.191235 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cddf54-c22f-4f9e-a9fc-b107b0555315 admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:44.193125 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cddf54-c22f-4f9e-a9fc-b107b0555315 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:47:44.194531 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cddf54-c22f-4f9e-a9fc-b107b0555315 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:44.200956 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cddf54-c22f-4f9e-a9fc-b107b0555315 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:44.251492 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 891/1774] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:47:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:47:44.256686 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52861ffc-6354-4c77-9623-4f96b41cfbef service placement] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:44.319981 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52861ffc-6354-4c77-9623-4f96b41cfbef service placement] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:44.334697 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516879a6-1a7f-451e-9c4f-0a39911ed72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:44.338555 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516879a6-1a7f-451e-9c4f-0a39911ed72b admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:44.338877 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516879a6-1a7f-451e-9c4f-0a39911ed72b admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:44.340145 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516879a6-1a7f-451e-9c4f-0a39911ed72b admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:44.340855 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516879a6-1a7f-451e-9c4f-0a39911ed72b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:44.350336 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516879a6-1a7f-451e-9c4f-0a39911ed72b admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:44.442840 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cddf54-c22f-4f9e-a9fc-b107b0555315 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:44.443743 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 884/1775] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:47:44 2022] POST /identity/v3/projects => generated 373 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:44.462856 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cddf54-c22f-4f9e-a9fc-b107b0555315 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:44.615815 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72661dc9-4366-4b1c-a456-702cff957e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:44.616574 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72661dc9-4366-4b1c-a456-702cff957e15 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:44.617884 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72661dc9-4366-4b1c-a456-702cff957e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:44.618326 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72661dc9-4366-4b1c-a456-702cff957e15 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:44.619135 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72661dc9-4366-4b1c-a456-702cff957e15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:44.625890 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72661dc9-4366-4b1c-a456-702cff957e15 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:44.675363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:44.688611 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516879a6-1a7f-451e-9c4f-0a39911ed72b admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:44.688721 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 892/1776] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:47:44 2022] POST /identity/v3/users => generated 287 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:47:44.724871 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516879a6-1a7f-451e-9c4f-0a39911ed72b admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:44.746942 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa0cb99-77f1-4aa8-b50d-262694c61f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:44.749080 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0cb99-77f1-4aa8-b50d-262694c61f8b admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:44.749080 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0cb99-77f1-4aa8-b50d-262694c61f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:44.765225 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0cb99-77f1-4aa8-b50d-262694c61f8b admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:44.767752 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa0cb99-77f1-4aa8-b50d-262694c61f8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:44.776768 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa0cb99-77f1-4aa8-b50d-262694c61f8b admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:44.826070 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 885/1777] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:44.832584 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72661dc9-4366-4b1c-a456-702cff957e15 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:44.853315 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72661dc9-4366-4b1c-a456-702cff957e15 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:44.879503 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8819437-141f-4d83-a5e8-64c99ccb6a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:44.880163 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8819437-141f-4d83-a5e8-64c99ccb6a1c admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:44.880447 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8819437-141f-4d83-a5e8-64c99ccb6a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:44.880722 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8819437-141f-4d83-a5e8-64c99ccb6a1c admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:44.890189 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8819437-141f-4d83-a5e8-64c99ccb6a1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:44.894335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8819437-141f-4d83-a5e8-64c99ccb6a1c admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:44.950695 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 893/1778] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:44.957964 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa0cb99-77f1-4aa8-b50d-262694c61f8b admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:44.980054 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa0cb99-77f1-4aa8-b50d-262694c61f8b admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:45.021268 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0d222-3583-49d4-922f-47b98772d5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:45.023078 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0d222-3583-49d4-922f-47b98772d5ec admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:45.023549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0d222-3583-49d4-922f-47b98772d5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:45.026684 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0d222-3583-49d4-922f-47b98772d5ec admin admin] PATH_INFO: `/v3/projects/1e1c8a118bcf4b9a9bf176a15a5cbb27/users/f20f56d7b013402bade36732ceae9bbc/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:45.089065 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f0d222-3583-49d4-922f-47b98772d5ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e1c8a118bcf4b9a9bf176a15a5cbb27, user_id=f20f56d7b013402bade36732ceae9bbc, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:45.103059 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f0d222-3583-49d4-922f-47b98772d5ec admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:45.157725 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 886/1779] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:47:44 2022] PUT /identity/v3/projects/1e1c8a118bcf4b9a9bf176a15a5cbb27/users/f20f56d7b013402bade36732ceae9bbc/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:45.169862 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df07d3-96dd-45a7-9393-6bae01162c11 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:45.170153 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df07d3-96dd-45a7-9393-6bae01162c11 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:45.170432 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df07d3-96dd-45a7-9393-6bae01162c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:45.249899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-07df07d3-96dd-45a7-9393-6bae01162c11 None None] MFA Rules not processed for user `f20f56d7b013402bade36732ceae9bbc`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:45.252188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-07df07d3-96dd-45a7-9393-6bae01162c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:45.396534 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0d222-3583-49d4-922f-47b98772d5ec admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:45.402645 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-e6f0d222-3583-49d4-922f-47b98772d5ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:45.525631 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 894/1780] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:45.542365 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df07d3-96dd-45a7-9393-6bae01162c11 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:45.544447 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-07df07d3-96dd-45a7-9393-6bae01162c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:45.615641 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:45.690942 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0d222-3583-49d4-922f-47b98772d5ec admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:45.707393 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253058b3-5dbc-46ae-9038-1cd869e5f768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:45.708667 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253058b3-5dbc-46ae-9038-1cd869e5f768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:45.708954 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253058b3-5dbc-46ae-9038-1cd869e5f768 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:45.713109 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253058b3-5dbc-46ae-9038-1cd869e5f768 admin admin] PATH_INFO: `/v3/projects/210742a89cc34af2bbc6eae9a3fd3a0e` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:45.713109 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253058b3-5dbc-46ae-9038-1cd869e5f768 admin admin] RBAC: Authorizing `identity:delete_project(project_id=210742a89cc34af2bbc6eae9a3fd3a0e)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:45.717237 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253058b3-5dbc-46ae-9038-1cd869e5f768 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:45.763524 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df07d3-96dd-45a7-9393-6bae01162c11 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:45.770617 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afd065-3da0-49f7-a604-3ccbacaa2223 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:45.771742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afd065-3da0-49f7-a604-3ccbacaa2223 service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:45.772043 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afd065-3da0-49f7-a604-3ccbacaa2223 service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:45.772907 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afd065-3da0-49f7-a604-3ccbacaa2223 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:45.818364 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2afd065-3da0-49f7-a604-3ccbacaa2223 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:45.820749 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2afd065-3da0-49f7-a604-3ccbacaa2223 service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:45.841832 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 895/1781] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 22:47:45.863495 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afd065-3da0-49f7-a604-3ccbacaa2223 service cinder] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:45.895623 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afd065-3da0-49f7-a604-3ccbacaa2223 service cinder] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:45.898811 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-253058b3-5dbc-46ae-9038-1cd869e5f768 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '210742a89cc34af2bbc6eae9a3fd3a0e', 'request_id': 'req-253058b3-5dbc-46ae-9038-1cd869e5f768'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:45.935397 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-253058b3-5dbc-46ae-9038-1cd869e5f768 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '210742a89cc34af2bbc6eae9a3fd3a0e', 'request_id': 'req-253058b3-5dbc-46ae-9038-1cd869e5f768'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:45.956317 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-253058b3-5dbc-46ae-9038-1cd869e5f768 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '210742a89cc34af2bbc6eae9a3fd3a0e', 'request_id': 'req-253058b3-5dbc-46ae-9038-1cd869e5f768'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:45.973488 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-253058b3-5dbc-46ae-9038-1cd869e5f768 admin admin] The token cache is being invalidate because project 210742a89cc34af2bbc6eae9a3fd3a0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:47:45.973931 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-253058b3-5dbc-46ae-9038-1cd869e5f768 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:45.976029 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-253058b3-5dbc-46ae-9038-1cd869e5f768 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:45.978714 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 887/1782] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:47:45 2022] DELETE /identity/v3/projects/210742a89cc34af2bbc6eae9a3fd3a0e => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:45.983240 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602201d-900d-40a2-85df-bb8951f83adf None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:45.985106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602201d-900d-40a2-85df-bb8951f83adf None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:45.986052 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602201d-900d-40a2-85df-bb8951f83adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:46.039083 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724ffa1-f51b-462f-aee6-a63d191b2870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:46.039702 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-7602201d-900d-40a2-85df-bb8951f83adf None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:46.039841 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724ffa1-f51b-462f-aee6-a63d191b2870 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:46.040123 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724ffa1-f51b-462f-aee6-a63d191b2870 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:46.040405 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724ffa1-f51b-462f-aee6-a63d191b2870 admin admin] PATH_INFO: `/v3/users/f20f56d7b013402bade36732ceae9bbc` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:46.042095 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-7602201d-900d-40a2-85df-bb8951f83adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:46.045544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1724ffa1-f51b-462f-aee6-a63d191b2870 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f20f56d7b013402bade36732ceae9bbc)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:46.049551 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1724ffa1-f51b-462f-aee6-a63d191b2870 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:46.076738 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 888/1783] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:46.083187 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa75d8-af77-4854-a3ab-0b537427bbbf None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:46.084417 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa75d8-af77-4854-a3ab-0b537427bbbf None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:46.085148 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa75d8-af77-4854-a3ab-0b537427bbbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:46.104391 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:46.150503 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-faaa75d8-af77-4854-a3ab-0b537427bbbf None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:46.151951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-faaa75d8-af77-4854-a3ab-0b537427bbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:46.164590 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 889/1784] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:46.165843 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-1724ffa1-f51b-462f-aee6-a63d191b2870 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f20f56d7b013402bade36732ceae9bbc'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:46.171660 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa75d8-af77-4854-a3ab-0b537427bbbf None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:46.185815 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-1724ffa1-f51b-462f-aee6-a63d191b2870 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f20f56d7b013402bade36732ceae9bbc'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:46.191567 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa75d8-af77-4854-a3ab-0b537427bbbf None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:46.198254 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-1724ffa1-f51b-462f-aee6-a63d191b2870 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f20f56d7b013402bade36732ceae9bbc'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:46.198980 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-1724ffa1-f51b-462f-aee6-a63d191b2870 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f20f56d7b013402bade36732ceae9bbc'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:46.199598 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-1724ffa1-f51b-462f-aee6-a63d191b2870 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f20f56d7b013402bade36732ceae9bbc'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:46.200165 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d66ca-8671-43d0-b4b5-5eb54323add9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:46.201570 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d66ca-8671-43d0-b4b5-5eb54323add9 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:46.202258 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d66ca-8671-43d0-b4b5-5eb54323add9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:46.202606 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d66ca-8671-43d0-b4b5-5eb54323add9 admin admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:46.203340 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25d66ca-8671-43d0-b4b5-5eb54323add9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:47:46.203764 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25d66ca-8671-43d0-b4b5-5eb54323add9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:46.207512 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25d66ca-8671-43d0-b4b5-5eb54323add9 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:46.232280 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 890/1785] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:47:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:47:46.233995 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d66ca-8671-43d0-b4b5-5eb54323add9 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:46.237422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-c25d66ca-8671-43d0-b4b5-5eb54323add9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:46.271301 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 896/1786] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:47:45 2022] DELETE /identity/v3/users/f20f56d7b013402bade36732ceae9bbc => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:47:46.275731 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e2ed8-a98a-4edb-8c19-bbb59b91fd1f None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:46.276253 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e2ed8-a98a-4edb-8c19-bbb59b91fd1f None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:46.276743 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e2ed8-a98a-4edb-8c19-bbb59b91fd1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:46.347316 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d66ca-8671-43d0-b4b5-5eb54323add9 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:46.350597 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-6c7e2ed8-a98a-4edb-8c19-bbb59b91fd1f None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:46.353495 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-6c7e2ed8-a98a-4edb-8c19-bbb59b91fd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:46.543443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26945b40-5e4b-41b8-a7a4-e44ed6d1249c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:46.544173 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26945b40-5e4b-41b8-a7a4-e44ed6d1249c admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:46.544488 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26945b40-5e4b-41b8-a7a4-e44ed6d1249c admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:46.544991 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26945b40-5e4b-41b8-a7a4-e44ed6d1249c admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:46.552385 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 897/1787] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:46.562635 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26945b40-5e4b-41b8-a7a4-e44ed6d1249c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:46.566450 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26945b40-5e4b-41b8-a7a4-e44ed6d1249c admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:46.587058 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e2ed8-a98a-4edb-8c19-bbb59b91fd1f None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:46.589422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-6c7e2ed8-a98a-4edb-8c19-bbb59b91fd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:46.622125 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 891/1788] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:47:46 2022] POST /identity/v3/projects => generated 385 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:46.626354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26945b40-5e4b-41b8-a7a4-e44ed6d1249c admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:46.673739 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:46.730157 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e2ed8-a98a-4edb-8c19-bbb59b91fd1f None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:46.741795 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26945b40-5e4b-41b8-a7a4-e44ed6d1249c admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:46.748978 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4610f6de-9d5b-40bc-80fc-68deca4e0d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:46.749798 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610f6de-9d5b-40bc-80fc-68deca4e0d1f admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:46.750137 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610f6de-9d5b-40bc-80fc-68deca4e0d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:46.750592 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610f6de-9d5b-40bc-80fc-68deca4e0d1f admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:46.751245 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4610f6de-9d5b-40bc-80fc-68deca4e0d1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:46.754085 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4610f6de-9d5b-40bc-80fc-68deca4e0d1f admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:46.798737 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 892/1789] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:47:46 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:46.799650 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4610f6de-9d5b-40bc-80fc-68deca4e0d1f admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:46.801605 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-4610f6de-9d5b-40bc-80fc-68deca4e0d1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:46.811269 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb743e-39bb-421b-93f4-c43428492daa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:46.812314 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb743e-39bb-421b-93f4-c43428492daa service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:46.812735 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb743e-39bb-421b-93f4-c43428492daa service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:46.813170 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb743e-39bb-421b-93f4-c43428492daa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:46.823045 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4610f6de-9d5b-40bc-80fc-68deca4e0d1f admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:46.825686 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb743e-39bb-421b-93f4-c43428492daa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:46.827666 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb743e-39bb-421b-93f4-c43428492daa service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:46.830185 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2b8bd-7ab9-4b23-bc09-754f8125617f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:46.830803 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2b8bd-7ab9-4b23-bc09-754f8125617f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:46.831279 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2b8bd-7ab9-4b23-bc09-754f8125617f admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:46.831618 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2b8bd-7ab9-4b23-bc09-754f8125617f admin admin] PATH_INFO: `/v3/projects/c1191dfdda0641d58b6792dd8053c891` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:46.832938 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c2b8bd-7ab9-4b23-bc09-754f8125617f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1191dfdda0641d58b6792dd8053c891)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:46.836051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c2b8bd-7ab9-4b23-bc09-754f8125617f admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:46.845242 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:46.853253 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 898/1790] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:46.854283 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb743e-39bb-421b-93f4-c43428492daa service neutron] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:46.867713 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb743e-39bb-421b-93f4-c43428492daa service neutron] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:46.878419 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498fc7d1-a750-45d4-b4a3-870c8c7d6fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:46.879260 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fc7d1-a750-45d4-b4a3-870c8c7d6fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:46.879609 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fc7d1-a750-45d4-b4a3-870c8c7d6fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:46.879882 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fc7d1-a750-45d4-b4a3-870c8c7d6fb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:46.880561 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498fc7d1-a750-45d4-b4a3-870c8c7d6fb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:46.883050 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498fc7d1-a750-45d4-b4a3-870c8c7d6fb1 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:46.907206 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 899/1791] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:46 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:46.907206 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498fc7d1-a750-45d4-b4a3-870c8c7d6fb1 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:46.920173 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498fc7d1-a750-45d4-b4a3-870c8c7d6fb1 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:46.925923 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc81be-e7d4-4b5a-89ff-79381384d491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:46.926819 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc81be-e7d4-4b5a-89ff-79381384d491 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:46.926819 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc81be-e7d4-4b5a-89ff-79381384d491 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:46.927114 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc81be-e7d4-4b5a-89ff-79381384d491 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:46.927787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dc81be-e7d4-4b5a-89ff-79381384d491 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:46.931718 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dc81be-e7d4-4b5a-89ff-79381384d491 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:46.948316 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 900/1792] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:46 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 22:47:46.951223 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb7b35-9231-414e-8556-f7603e2f7d75 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:46.951540 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb7b35-9231-414e-8556-f7603e2f7d75 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:46.952248 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb7b35-9231-414e-8556-f7603e2f7d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:47.004817 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-12c2b8bd-7ab9-4b23-bc09-754f8125617f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1191dfdda0641d58b6792dd8053c891', 'request_id': 'req-12c2b8bd-7ab9-4b23-bc09-754f8125617f'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:47.019183 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-84bb7b35-9231-414e-8556-f7603e2f7d75 None None] MFA Rules not processed for user `b874dd1d3675426ea36bc55e7c660c85`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:47.020581 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-84bb7b35-9231-414e-8556-f7603e2f7d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:47.026563 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-12c2b8bd-7ab9-4b23-bc09-754f8125617f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1191dfdda0641d58b6792dd8053c891', 'request_id': 'req-12c2b8bd-7ab9-4b23-bc09-754f8125617f'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:47.044002 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-12c2b8bd-7ab9-4b23-bc09-754f8125617f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1191dfdda0641d58b6792dd8053c891', 'request_id': 'req-12c2b8bd-7ab9-4b23-bc09-754f8125617f'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:47.080274 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-12c2b8bd-7ab9-4b23-bc09-754f8125617f admin admin] The token cache is being invalidate because project c1191dfdda0641d58b6792dd8053c891 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:47:47.084408 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-12c2b8bd-7ab9-4b23-bc09-754f8125617f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:47.088466 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-12c2b8bd-7ab9-4b23-bc09-754f8125617f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:47.090802 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 893/1793] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:47:46 2022] DELETE /identity/v3/projects/c1191dfdda0641d58b6792dd8053c891 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:47.095144 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2b8bd-7ab9-4b23-bc09-754f8125617f admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:47.098164 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-12c2b8bd-7ab9-4b23-bc09-754f8125617f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:47.208679 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 901/1794] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:46 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:47.221903 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecedbee-4988-45e2-85f6-dc61367118b2 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:47.222214 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecedbee-4988-45e2-85f6-dc61367118b2 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:47.222482 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecedbee-4988-45e2-85f6-dc61367118b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:47.250581 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-9ecedbee-4988-45e2-85f6-dc61367118b2 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:47.252359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-9ecedbee-4988-45e2-85f6-dc61367118b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:47.336357 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2b8bd-7ab9-4b23-bc09-754f8125617f admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:47.357020 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07baf74-6a39-4362-820e-449719dcd7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:47.358716 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07baf74-6a39-4362-820e-449719dcd7cd admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:47.359956 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07baf74-6a39-4362-820e-449719dcd7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:47.361696 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07baf74-6a39-4362-820e-449719dcd7cd admin admin] PATH_INFO: `/v3/projects/a55e5a8aae6e497db0fc4be4347021e0/users/fa893fb96e354b76b3a6b59f3e4249f7/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:47.417971 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07baf74-6a39-4362-820e-449719dcd7cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a55e5a8aae6e497db0fc4be4347021e0, user_id=fa893fb96e354b76b3a6b59f3e4249f7, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:47.421471 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07baf74-6a39-4362-820e-449719dcd7cd admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:47.450161 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 894/1795] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:47:47 2022] PUT /identity/v3/projects/a55e5a8aae6e497db0fc4be4347021e0/users/fa893fb96e354b76b3a6b59f3e4249f7/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:47.451531 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07baf74-6a39-4362-820e-449719dcd7cd admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:47.453312 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-b07baf74-6a39-4362-820e-449719dcd7cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:47.510730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:47.559936 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 902/1796] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:47.561174 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f591d6-4a0e-465b-ab68-a4ee65314037 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:47.561543 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f591d6-4a0e-465b-ab68-a4ee65314037 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:47.561824 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f591d6-4a0e-465b-ab68-a4ee65314037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:47.603106 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-13f591d6-4a0e-465b-ab68-a4ee65314037 None None] MFA Rules not processed for user `fa893fb96e354b76b3a6b59f3e4249f7`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:47.604640 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-13f591d6-4a0e-465b-ab68-a4ee65314037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:47.686693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07baf74-6a39-4362-820e-449719dcd7cd admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:47.692608 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ea267-d6d5-4fed-ad1c-2c0d8310c7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:47.694487 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ea267-d6d5-4fed-ad1c-2c0d8310c7a3 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:47.695265 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ea267-d6d5-4fed-ad1c-2c0d8310c7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:47.696054 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ea267-d6d5-4fed-ad1c-2c0d8310c7a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:47.696938 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ea267-d6d5-4fed-ad1c-2c0d8310c7a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:47.700008 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ea267-d6d5-4fed-ad1c-2c0d8310c7a3 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:47.758324 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 895/1797] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:47:47 2022] POST /identity/v3/projects => generated 403 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:47.759150 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab653e-e3d8-4e59-918d-2e1dfe97f4e9 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:47.759345 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab653e-e3d8-4e59-918d-2e1dfe97f4e9 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:47.759610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab653e-e3d8-4e59-918d-2e1dfe97f4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:47.799385 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:47.810543 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-9cab653e-e3d8-4e59-918d-2e1dfe97f4e9 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:47.812017 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-9cab653e-e3d8-4e59-918d-2e1dfe97f4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:47.815185 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 903/1798] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:47.816066 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f591d6-4a0e-465b-ab68-a4ee65314037 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:47.845690 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f591d6-4a0e-465b-ab68-a4ee65314037 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:47.996047 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 896/1799] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:47.997163 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b1418-7a95-4135-a6c7-88651bb60370 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:48.002391 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b1418-7a95-4135-a6c7-88651bb60370 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:48.002730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b1418-7a95-4135-a6c7-88651bb60370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:48.034856 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d603fd-81a1-49d8-8d9b-32d566750b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:48.036705 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d603fd-81a1-49d8-8d9b-32d566750b9c admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:48.039699 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d603fd-81a1-49d8-8d9b-32d566750b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:48.041912 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d603fd-81a1-49d8-8d9b-32d566750b9c admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:48.044023 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d603fd-81a1-49d8-8d9b-32d566750b9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:48.053110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d603fd-81a1-49d8-8d9b-32d566750b9c admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:48.107438 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 904/1800] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:47:47 2022] POST /identity/v3/users => generated 308 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:48.111243 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d603fd-81a1-49d8-8d9b-32d566750b9c admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:48.121925 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-a28b1418-7a95-4135-a6c7-88651bb60370 None None] MFA Rules not processed for user `aff5e804c64345b298c745287a1d0a32`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:48.123518 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-a28b1418-7a95-4135-a6c7-88651bb60370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:48.140835 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d603fd-81a1-49d8-8d9b-32d566750b9c admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:48.155840 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c19e4-5cf3-444e-82bd-e9c4314c1b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:48.157999 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c19e4-5cf3-444e-82bd-e9c4314c1b9e admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:48.159731 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c19e4-5cf3-444e-82bd-e9c4314c1b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:48.161693 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c19e4-5cf3-444e-82bd-e9c4314c1b9e admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:48.165412 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c19e4-5cf3-444e-82bd-e9c4314c1b9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:48.171462 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c19e4-5cf3-444e-82bd-e9c4314c1b9e admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:48.237464 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 905/1801] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:47:48 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:48.238819 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c19e4-5cf3-444e-82bd-e9c4314c1b9e admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:48.240106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 897/1802] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 10 22:47:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:48.240868 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b1418-7a95-4135-a6c7-88651bb60370 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:48.272402 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b1418-7a95-4135-a6c7-88651bb60370 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:48.313480 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:48.333154 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c19e4-5cf3-444e-82bd-e9c4314c1b9e admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:48.339446 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4f0e24-c3a7-494a-9233-f09eb213347f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:48.340440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f0e24-c3a7-494a-9233-f09eb213347f admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:48.340833 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f0e24-c3a7-494a-9233-f09eb213347f admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:48.341217 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f0e24-c3a7-494a-9233-f09eb213347f admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:48.342016 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4f0e24-c3a7-494a-9233-f09eb213347f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:47:48.343294 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4f0e24-c3a7-494a-9233-f09eb213347f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:48.345580 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4f0e24-c3a7-494a-9233-f09eb213347f admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:48.364644 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 906/1803] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:47:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:47:48.364644 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4f0e24-c3a7-494a-9233-f09eb213347f admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:48.378761 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4f0e24-c3a7-494a-9233-f09eb213347f admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:48.384519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab212ca-48f9-4306-be4a-fe7459f61b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:48.385077 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79679599-9c07-4575-9ebe-9dcd52cea555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:48.385715 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab212ca-48f9-4306-be4a-fe7459f61b56 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:48.385844 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79679599-9c07-4575-9ebe-9dcd52cea555 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:48.386011 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79679599-9c07-4575-9ebe-9dcd52cea555 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:48.386288 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79679599-9c07-4575-9ebe-9dcd52cea555 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:48.386397 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab212ca-48f9-4306-be4a-fe7459f61b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:48.386743 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab212ca-48f9-4306-be4a-fe7459f61b56 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:48.386941 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79679599-9c07-4575-9ebe-9dcd52cea555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:48.388200 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab212ca-48f9-4306-be4a-fe7459f61b56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:48.388565 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79679599-9c07-4575-9ebe-9dcd52cea555 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:48.390747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab212ca-48f9-4306-be4a-fe7459f61b56 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:48.417582 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 898/1804] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:48 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:48.418455 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79679599-9c07-4575-9ebe-9dcd52cea555 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:48.439083 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 907/1805] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:47:48 2022] POST /identity/v3/users => generated 299 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:48.439083 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab212ca-48f9-4306-be4a-fe7459f61b56 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:48.440823 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79679599-9c07-4575-9ebe-9dcd52cea555 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:48.456031 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd92b9c-6313-4e0c-bf77-2fa0ffa0ae13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:48.456751 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd92b9c-6313-4e0c-bf77-2fa0ffa0ae13 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:48.457039 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd92b9c-6313-4e0c-bf77-2fa0ffa0ae13 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:48.457336 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd92b9c-6313-4e0c-bf77-2fa0ffa0ae13 admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:48.457975 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd92b9c-6313-4e0c-bf77-2fa0ffa0ae13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:48.460043 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab212ca-48f9-4306-be4a-fe7459f61b56 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:48.460234 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd92b9c-6313-4e0c-bf77-2fa0ffa0ae13 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:48.468621 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ba2a3-f25e-4cca-bf9b-6baaded0a2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:48.469810 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ba2a3-f25e-4cca-bf9b-6baaded0a2bf admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:48.470317 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ba2a3-f25e-4cca-bf9b-6baaded0a2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:48.471853 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ba2a3-f25e-4cca-bf9b-6baaded0a2bf admin admin] PATH_INFO: `/v3/projects/2378d661540746018725207421ec14fd/users/3965cfcfffc348a98307954f2af92b5b/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:48.516557 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09ba2a3-f25e-4cca-bf9b-6baaded0a2bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2378d661540746018725207421ec14fd, user_id=3965cfcfffc348a98307954f2af92b5b, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:48.520732 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09ba2a3-f25e-4cca-bf9b-6baaded0a2bf admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:48.529889 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 899/1806] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:47:48 2022] POST /identity/v3/projects => generated 381 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:48.529889 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd92b9c-6313-4e0c-bf77-2fa0ffa0ae13 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:48.597265 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 908/1807] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:47:48 2022] PUT /identity/v3/projects/2378d661540746018725207421ec14fd/users/3965cfcfffc348a98307954f2af92b5b/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:48.597265 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ba2a3-f25e-4cca-bf9b-6baaded0a2bf admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:48.605156 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-a09ba2a3-f25e-4cca-bf9b-6baaded0a2bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:48.681040 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd92b9c-6313-4e0c-bf77-2fa0ffa0ae13 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:48.682286 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:48.757134 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfa41ed-655a-4119-a2a5-4c644cba5cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:48.757803 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa41ed-655a-4119-a2a5-4c644cba5cf9 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:48.758134 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa41ed-655a-4119-a2a5-4c644cba5cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:48.758408 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa41ed-655a-4119-a2a5-4c644cba5cf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:48.759058 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfa41ed-655a-4119-a2a5-4c644cba5cf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:48.761136 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfa41ed-655a-4119-a2a5-4c644cba5cf9 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:48.778439 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 900/1808] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:48 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:48.778439 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfa41ed-655a-4119-a2a5-4c644cba5cf9 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:48.792049 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfa41ed-655a-4119-a2a5-4c644cba5cf9 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:48.799638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35704479-cc9d-4129-b6a3-03bca6fd71ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:48.800392 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35704479-cc9d-4129-b6a3-03bca6fd71ef admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:48.800671 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35704479-cc9d-4129-b6a3-03bca6fd71ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:48.800997 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35704479-cc9d-4129-b6a3-03bca6fd71ef admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:48.801862 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35704479-cc9d-4129-b6a3-03bca6fd71ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:48.805827 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35704479-cc9d-4129-b6a3-03bca6fd71ef admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:48.844932 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 901/1809] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:47:48 2022] POST /identity/v3/users => generated 291 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:48.844932 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35704479-cc9d-4129-b6a3-03bca6fd71ef admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:48.873558 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35704479-cc9d-4129-b6a3-03bca6fd71ef admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:48.876344 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ba2a3-f25e-4cca-bf9b-6baaded0a2bf admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:48.882710 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188c1a8-3ec7-4dad-9ad4-67292517b664 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e3a76df95a34b9fb9816f22b85d1071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:48.884199 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188c1a8-3ec7-4dad-9ad4-67292517b664 service swift] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:48.885150 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e3260d-7c30-4583-b64a-cb2e3f5924a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:48.885568 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188c1a8-3ec7-4dad-9ad4-67292517b664 service swift] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:48.885927 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188c1a8-3ec7-4dad-9ad4-67292517b664 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:48.887030 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3260d-7c30-4583-b64a-cb2e3f5924a4 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:48.887030 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3260d-7c30-4583-b64a-cb2e3f5924a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:48.887514 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3260d-7c30-4583-b64a-cb2e3f5924a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:48.889034 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e3260d-7c30-4583-b64a-cb2e3f5924a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:48.894940 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e3260d-7c30-4583-b64a-cb2e3f5924a4 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:48.959994 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 902/1810] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:48 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:48.961316 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e3260d-7c30-4583-b64a-cb2e3f5924a4 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:48.964710 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-43e3260d-7c30-4583-b64a-cb2e3f5924a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:48.996152 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e3260d-7c30-4583-b64a-cb2e3f5924a4 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:49.008637 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52da7b-db0e-4c0e-9f92-0fc81d9a9791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:49.009260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52da7b-db0e-4c0e-9f92-0fc81d9a9791 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:49.009582 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52da7b-db0e-4c0e-9f92-0fc81d9a9791 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:49.009845 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52da7b-db0e-4c0e-9f92-0fc81d9a9791 admin admin] PATH_INFO: `/v3/projects/1e1c8a118bcf4b9a9bf176a15a5cbb27` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:49.016226 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d52da7b-db0e-4c0e-9f92-0fc81d9a9791 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e1c8a118bcf4b9a9bf176a15a5cbb27)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:49.021391 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1188c1a8-3ec7-4dad-9ad4-67292517b664 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:49.024411 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d52da7b-db0e-4c0e-9f92-0fc81d9a9791 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:49.026153 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1188c1a8-3ec7-4dad-9ad4-67292517b664 service swift] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:49.027178 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:49.048000 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 909/1811] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 10 22:47:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 22:47:49.048969 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188c1a8-3ec7-4dad-9ad4-67292517b664 service swift] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:49.080250 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188c1a8-3ec7-4dad-9ad4-67292517b664 service swift] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:49.090772 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e0993-e2fb-4039-abd6-1ff23ac5a7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:49.091478 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e0993-e2fb-4039-abd6-1ff23ac5a7d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:49.091762 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e0993-e2fb-4039-abd6-1ff23ac5a7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:49.092135 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e0993-e2fb-4039-abd6-1ff23ac5a7d3 admin admin] PATH_INFO: `/v3/projects/a22a97cbd7ae4efe9cd3b7d7206ead52/users/47491921214d46559d97c8bd5335c45d/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:49.128698 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e0993-e2fb-4039-abd6-1ff23ac5a7d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a22a97cbd7ae4efe9cd3b7d7206ead52, user_id=47491921214d46559d97c8bd5335c45d, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:49.132087 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e0993-e2fb-4039-abd6-1ff23ac5a7d3 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:49.170145 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 910/1812] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:47:49 2022] PUT /identity/v3/projects/a22a97cbd7ae4efe9cd3b7d7206ead52/users/47491921214d46559d97c8bd5335c45d/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:49.170904 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e0993-e2fb-4039-abd6-1ff23ac5a7d3 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:49.228339 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-9d52da7b-db0e-4c0e-9f92-0fc81d9a9791 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e1c8a118bcf4b9a9bf176a15a5cbb27', 'request_id': 'req-9d52da7b-db0e-4c0e-9f92-0fc81d9a9791'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:49.259863 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-9d52da7b-db0e-4c0e-9f92-0fc81d9a9791 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e1c8a118bcf4b9a9bf176a15a5cbb27', 'request_id': 'req-9d52da7b-db0e-4c0e-9f92-0fc81d9a9791'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:49.282452 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-9d52da7b-db0e-4c0e-9f92-0fc81d9a9791 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e1c8a118bcf4b9a9bf176a15a5cbb27', 'request_id': 'req-9d52da7b-db0e-4c0e-9f92-0fc81d9a9791'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:49.327852 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-9d52da7b-db0e-4c0e-9f92-0fc81d9a9791 admin admin] The token cache is being invalidate because project 1e1c8a118bcf4b9a9bf176a15a5cbb27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:47:49.328197 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-9d52da7b-db0e-4c0e-9f92-0fc81d9a9791 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:49.330436 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-9d52da7b-db0e-4c0e-9f92-0fc81d9a9791 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:49.330436 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 903/1813] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:47:48 2022] DELETE /identity/v3/projects/1e1c8a118bcf4b9a9bf176a15a5cbb27 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:49.347971 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52da7b-db0e-4c0e-9f92-0fc81d9a9791 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:49.354965 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-9d52da7b-db0e-4c0e-9f92-0fc81d9a9791 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:49.377626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e0993-e2fb-4039-abd6-1ff23ac5a7d3 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:49.450442 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:49.629050 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da710568-407f-47c4-a476-8d9621f9e944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:49.638556 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da710568-407f-47c4-a476-8d9621f9e944 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:49.638936 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da710568-407f-47c4-a476-8d9621f9e944 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:49.639178 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da710568-407f-47c4-a476-8d9621f9e944 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:49.644857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da710568-407f-47c4-a476-8d9621f9e944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:49.646867 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da710568-407f-47c4-a476-8d9621f9e944 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:49.647044 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52da7b-db0e-4c0e-9f92-0fc81d9a9791 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:49.664393 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e6af3e-c16b-46f8-ba43-7c4c6a4dd85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:49.665099 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6af3e-c16b-46f8-ba43-7c4c6a4dd85f admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:49.665448 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6af3e-c16b-46f8-ba43-7c4c6a4dd85f admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:49.665765 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6af3e-c16b-46f8-ba43-7c4c6a4dd85f admin admin] PATH_INFO: `/v3/projects/2378d661540746018725207421ec14fd/users/3965cfcfffc348a98307954f2af92b5b/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:49.673520 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e6af3e-c16b-46f8-ba43-7c4c6a4dd85f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2378d661540746018725207421ec14fd, user_id=3965cfcfffc348a98307954f2af92b5b, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:49.688599 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e6af3e-c16b-46f8-ba43-7c4c6a4dd85f admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:49.707569 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 911/1814] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 535 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:49.709206 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080cdbb9-f104-49e9-bb36-75cdedae30b6 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:49.709521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080cdbb9-f104-49e9-bb36-75cdedae30b6 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:49.715253 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080cdbb9-f104-49e9-bb36-75cdedae30b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:49.729970 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 904/1815] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:47:49 2022] PUT /identity/v3/projects/2378d661540746018725207421ec14fd/users/3965cfcfffc348a98307954f2af92b5b/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:49.731026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e6af3e-c16b-46f8-ba43-7c4c6a4dd85f admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:49.742455 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-c5e6af3e-c16b-46f8-ba43-7c4c6a4dd85f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:49.812355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-080cdbb9-f104-49e9-bb36-75cdedae30b6 None None] MFA Rules not processed for user `da22fc66a1d14fa983463331de110258`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:49.814512 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-080cdbb9-f104-49e9-bb36-75cdedae30b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:49.983814 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e6af3e-c16b-46f8-ba43-7c4c6a4dd85f admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:50.001659 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795c8ab-48ad-4168-8bd4-8546c7d18b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:50.004985 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795c8ab-48ad-4168-8bd4-8546c7d18b47 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:50.006215 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795c8ab-48ad-4168-8bd4-8546c7d18b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:50.007690 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795c8ab-48ad-4168-8bd4-8546c7d18b47 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:50.009038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b795c8ab-48ad-4168-8bd4-8546c7d18b47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:50.013881 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b795c8ab-48ad-4168-8bd4-8546c7d18b47 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:50.098564 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 905/1816] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:50.110117 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437c0a5-314e-483a-aac0-bdc599997169 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:50.110969 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437c0a5-314e-483a-aac0-bdc599997169 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:50.112292 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437c0a5-314e-483a-aac0-bdc599997169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:50.114676 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 912/1817] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 408 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:50.124132 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080cdbb9-f104-49e9-bb36-75cdedae30b6 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:50.126138 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-080cdbb9-f104-49e9-bb36-75cdedae30b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:50.171310 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080cdbb9-f104-49e9-bb36-75cdedae30b6 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:50.207826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c643f60-a125-482b-8cfe-c844ac160743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:50.211677 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c643f60-a125-482b-8cfe-c844ac160743 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:50.217741 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c643f60-a125-482b-8cfe-c844ac160743 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:50.229321 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c643f60-a125-482b-8cfe-c844ac160743 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:50.229321 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c643f60-a125-482b-8cfe-c844ac160743 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:50.238469 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c643f60-a125-482b-8cfe-c844ac160743 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:50.307922 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 913/1818] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:50 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:50.309561 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c643f60-a125-482b-8cfe-c844ac160743 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:50.353678 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c643f60-a125-482b-8cfe-c844ac160743 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:50.375283 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-6437c0a5-314e-483a-aac0-bdc599997169 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:50.387932 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-6437c0a5-314e-483a-aac0-bdc599997169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:50.392991 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489fd778-eda2-4feb-b3cc-d6eca5897209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:50.394107 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489fd778-eda2-4feb-b3cc-d6eca5897209 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:50.394578 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489fd778-eda2-4feb-b3cc-d6eca5897209 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:50.395048 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489fd778-eda2-4feb-b3cc-d6eca5897209 admin admin] PATH_INFO: `/v3/projects/a22a97cbd7ae4efe9cd3b7d7206ead52/users/47491921214d46559d97c8bd5335c45d/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:50.402642 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489fd778-eda2-4feb-b3cc-d6eca5897209 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a22a97cbd7ae4efe9cd3b7d7206ead52, user_id=47491921214d46559d97c8bd5335c45d, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:50.414886 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489fd778-eda2-4feb-b3cc-d6eca5897209 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:50.467924 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 906/1819] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:50.475180 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437c0a5-314e-483a-aac0-bdc599997169 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:50.480455 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 914/1820] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:47:50 2022] PUT /identity/v3/projects/a22a97cbd7ae4efe9cd3b7d7206ead52/users/47491921214d46559d97c8bd5335c45d/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:50.495506 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392bfd2-14b6-41fc-8c50-969b53f9ee06 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:50.495968 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392bfd2-14b6-41fc-8c50-969b53f9ee06 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:50.496423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392bfd2-14b6-41fc-8c50-969b53f9ee06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:50.566526 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437c0a5-314e-483a-aac0-bdc599997169 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:50.603001 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:50.642318 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-4392bfd2-14b6-41fc-8c50-969b53f9ee06 None None] MFA Rules not processed for user `7fcb5a674ff94b3d8f79efbbe4959069`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:50.650361 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-4392bfd2-14b6-41fc-8c50-969b53f9ee06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:50.886605 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608cbb8b-a64a-4c70-9b7d-5acf00250847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:50.888470 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cbb8b-a64a-4c70-9b7d-5acf00250847 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:50.889125 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cbb8b-a64a-4c70-9b7d-5acf00250847 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:50.890343 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cbb8b-a64a-4c70-9b7d-5acf00250847 admin admin] PATH_INFO: `/v3/projects/06a020c9f2f24235b4d728574d5a8127/users/7157eb675616440294a4bced8239e173/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:50.955282 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 915/1821] 10.222.0.95 () {58 vars in 1015 bytes} [Wed Aug 10 22:47:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:50.958900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12908c75-cfc3-473c-8311-154ab144ccea None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:50.961582 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12908c75-cfc3-473c-8311-154ab144ccea None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:50.963049 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608cbb8b-a64a-4c70-9b7d-5acf00250847 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06a020c9f2f24235b4d728574d5a8127, user_id=7157eb675616440294a4bced8239e173, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:50.963562 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12908c75-cfc3-473c-8311-154ab144ccea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:50.980599 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608cbb8b-a64a-4c70-9b7d-5acf00250847 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:51.017382 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 907/1822] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:47:50 2022] PUT /identity/v3/projects/06a020c9f2f24235b4d728574d5a8127/users/7157eb675616440294a4bced8239e173/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:51.023420 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7dd33-91a1-4a69-aa94-011b33bcc5f2 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:51.023897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7dd33-91a1-4a69-aa94-011b33bcc5f2 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:51.024205 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7dd33-91a1-4a69-aa94-011b33bcc5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:51.049091 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-12908c75-cfc3-473c-8311-154ab144ccea None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:51.057215 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-12908c75-cfc3-473c-8311-154ab144ccea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:51.130580 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-5db7dd33-91a1-4a69-aa94-011b33bcc5f2 None None] MFA Rules not processed for user `47491921214d46559d97c8bd5335c45d`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:51.132477 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-5db7dd33-91a1-4a69-aa94-011b33bcc5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:51.204231 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:51.289702 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 916/1823] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:51.292449 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b178cca9-cd75-40dc-908f-08415c12d4ea None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:51.294446 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b178cca9-cd75-40dc-908f-08415c12d4ea None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:51.295319 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b178cca9-cd75-40dc-908f-08415c12d4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:51.379770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-b178cca9-cd75-40dc-908f-08415c12d4ea None None] MFA Rules not processed for user `7157eb675616440294a4bced8239e173`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:51.381117 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-b178cca9-cd75-40dc-908f-08415c12d4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:51.387970 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 908/1824] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:51 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:51.392051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7dd33-91a1-4a69-aa94-011b33bcc5f2 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:51.394936 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-5db7dd33-91a1-4a69-aa94-011b33bcc5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:51.598967 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 917/1825] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:51.600163 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b178cca9-cd75-40dc-908f-08415c12d4ea None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:51.635926 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b178cca9-cd75-40dc-908f-08415c12d4ea None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:51.640497 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7dd33-91a1-4a69-aa94-011b33bcc5f2 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:51.645298 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5c5f33-f2bd-48ca-993f-2ac393944f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:51.646241 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c5f33-f2bd-48ca-993f-2ac393944f60 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:51.647376 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c5f33-f2bd-48ca-993f-2ac393944f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:51.648268 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c5f33-f2bd-48ca-993f-2ac393944f60 admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:51.652631 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124f5a6-e061-4c3c-ba8b-0d25a4e4b4db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:51.658708 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5c5f33-f2bd-48ca-993f-2ac393944f60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:47:51.659172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124f5a6-e061-4c3c-ba8b-0d25a4e4b4db service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:51.659566 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124f5a6-e061-4c3c-ba8b-0d25a4e4b4db service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:51.659842 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124f5a6-e061-4c3c-ba8b-0d25a4e4b4db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:51.660910 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5c5f33-f2bd-48ca-993f-2ac393944f60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:51.662594 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5c5f33-f2bd-48ca-993f-2ac393944f60 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:51.686027 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6124f5a6-e061-4c3c-ba8b-0d25a4e4b4db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:51.689542 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6124f5a6-e061-4c3c-ba8b-0d25a4e4b4db service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:51.708952 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 918/1826] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:47:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:47:51.710674 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5c5f33-f2bd-48ca-993f-2ac393944f60 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:51.716782 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-df5c5f33-f2bd-48ca-993f-2ac393944f60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:51.721958 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 909/1827] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:51.726134 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124f5a6-e061-4c3c-ba8b-0d25a4e4b4db service neutron] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:51.767602 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124f5a6-e061-4c3c-ba8b-0d25a4e4b4db service neutron] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:51.781967 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcefb80-3cf9-4a10-9509-fa8473ad55bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:51.782624 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcefb80-3cf9-4a10-9509-fa8473ad55bd admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:51.783110 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcefb80-3cf9-4a10-9509-fa8473ad55bd admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:51.783393 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcefb80-3cf9-4a10-9509-fa8473ad55bd admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:51.784027 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcefb80-3cf9-4a10-9509-fa8473ad55bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:51.790352 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcefb80-3cf9-4a10-9509-fa8473ad55bd admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:51.854390 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 910/1828] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:47:51 2022] POST /identity/v3/projects => generated 383 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:51.856739 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcefb80-3cf9-4a10-9509-fa8473ad55bd admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:51.883563 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:51.905975 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5c5f33-f2bd-48ca-993f-2ac393944f60 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:52.020553 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcefb80-3cf9-4a10-9509-fa8473ad55bd admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:52.029256 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797e718-44d6-4be7-9841-4e7a3cc1e433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:52.030209 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797e718-44d6-4be7-9841-4e7a3cc1e433 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:52.030494 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797e718-44d6-4be7-9841-4e7a3cc1e433 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:52.030771 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797e718-44d6-4be7-9841-4e7a3cc1e433 admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:52.031411 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2797e718-44d6-4be7-9841-4e7a3cc1e433 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:52.035344 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2797e718-44d6-4be7-9841-4e7a3cc1e433 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:52.106386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006f03f-fcf8-40cb-9a59-bd6e5605961a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:52.107108 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006f03f-fcf8-40cb-9a59-bd6e5605961a service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:52.107580 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006f03f-fcf8-40cb-9a59-bd6e5605961a service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:52.108224 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006f03f-fcf8-40cb-9a59-bd6e5605961a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:52.113312 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-e006f03f-fcf8-40cb-9a59-bd6e5605961a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:52.137929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 911/1829] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:47:51 2022] POST /identity/v3/projects => generated 383 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:52.138696 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797e718-44d6-4be7-9841-4e7a3cc1e433 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:52.277200 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:52.283939 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797e718-44d6-4be7-9841-4e7a3cc1e433 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:52.310754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e12a0-5a66-4501-8606-1d8692558cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:52.315492 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e12a0-5a66-4501-8606-1d8692558cb8 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:52.315926 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e12a0-5a66-4501-8606-1d8692558cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:52.316222 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e12a0-5a66-4501-8606-1d8692558cb8 admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:52.320414 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e12a0-5a66-4501-8606-1d8692558cb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:52.340073 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e12a0-5a66-4501-8606-1d8692558cb8 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:52.395087 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e006f03f-fcf8-40cb-9a59-bd6e5605961a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:52.402745 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e006f03f-fcf8-40cb-9a59-bd6e5605961a service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:52.434408 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 912/1830] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:47:52 2022] POST /identity/v3/users => generated 298 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:52.445843 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5146b88-4004-4886-abed-2909b549a381 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:52.446284 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5146b88-4004-4886-abed-2909b549a381 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:52.449949 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5146b88-4004-4886-abed-2909b549a381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:52.473515 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 919/1831] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 764 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:52.474446 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006f03f-fcf8-40cb-9a59-bd6e5605961a service cinder] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:52.519759 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006f03f-fcf8-40cb-9a59-bd6e5605961a service cinder] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:52.535245 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-c5146b88-4004-4886-abed-2909b549a381 None None] MFA Rules not processed for user `405738ed219b4fdc8e3c07706a13e5a2`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:52.536517 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-c5146b88-4004-4886-abed-2909b549a381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:52.552417 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ebe9f-3c24-4b26-999f-8f71703b92ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:52.553678 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ebe9f-3c24-4b26-999f-8f71703b92ae admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:52.555782 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ebe9f-3c24-4b26-999f-8f71703b92ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:52.557378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ebe9f-3c24-4b26-999f-8f71703b92ae admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:52.558188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ebe9f-3c24-4b26-999f-8f71703b92ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:52.566760 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ebe9f-3c24-4b26-999f-8f71703b92ae admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:52.621416 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 920/1832] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:47:52 2022] POST /identity/v3/users => generated 292 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:47:52.623812 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ebe9f-3c24-4b26-999f-8f71703b92ae admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:52.637975 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ebe9f-3c24-4b26-999f-8f71703b92ae admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:52.647533 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60bf73d-3341-4f82-add2-c3290c7f1a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:52.649129 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60bf73d-3341-4f82-add2-c3290c7f1a9f admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:52.649776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60bf73d-3341-4f82-add2-c3290c7f1a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:52.650298 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60bf73d-3341-4f82-add2-c3290c7f1a9f admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:52.651730 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60bf73d-3341-4f82-add2-c3290c7f1a9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:52.654372 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60bf73d-3341-4f82-add2-c3290c7f1a9f admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:52.673410 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 921/1833] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:52.674846 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60bf73d-3341-4f82-add2-c3290c7f1a9f admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:52.688763 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60bf73d-3341-4f82-add2-c3290c7f1a9f admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:52.697165 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b26aa93-cc23-4b68-9505-b99dda46b50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:52.698356 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26aa93-cc23-4b68-9505-b99dda46b50a admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:52.698923 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26aa93-cc23-4b68-9505-b99dda46b50a admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:52.699786 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26aa93-cc23-4b68-9505-b99dda46b50a admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:52.701056 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b26aa93-cc23-4b68-9505-b99dda46b50a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:52.703067 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b26aa93-cc23-4b68-9505-b99dda46b50a admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:52.720840 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 922/1834] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:52.722720 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b26aa93-cc23-4b68-9505-b99dda46b50a admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:52.737389 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b26aa93-cc23-4b68-9505-b99dda46b50a admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:52.747105 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edde230-dbd4-4f64-9dab-3e599aa95e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:52.748193 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edde230-dbd4-4f64-9dab-3e599aa95e32 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:52.748705 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edde230-dbd4-4f64-9dab-3e599aa95e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:52.749579 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edde230-dbd4-4f64-9dab-3e599aa95e32 admin admin] PATH_INFO: `/v3/projects/624fd3f243d9417198334938e17b6483/users/cf01e67fe4af4aada544c6505ee4e3b9/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:52.775360 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edde230-dbd4-4f64-9dab-3e599aa95e32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=624fd3f243d9417198334938e17b6483, user_id=cf01e67fe4af4aada544c6505ee4e3b9, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:52.781068 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edde230-dbd4-4f64-9dab-3e599aa95e32 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:52.808662 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 923/1835] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:47:52 2022] PUT /identity/v3/projects/624fd3f243d9417198334938e17b6483/users/cf01e67fe4af4aada544c6505ee4e3b9/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:52.810533 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 913/1836] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:52.811179 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edde230-dbd4-4f64-9dab-3e599aa95e32 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:52.815096 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5146b88-4004-4886-abed-2909b549a381 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:52.859155 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5146b88-4004-4886-abed-2909b549a381 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:52.984544 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:53.099423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edde230-dbd4-4f64-9dab-3e599aa95e32 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:53.110348 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee0818d-0665-461c-bda5-5c0c451b634e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:53.111645 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109791d6-a982-4967-9e97-f55f58031199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:53.111991 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee0818d-0665-461c-bda5-5c0c451b634e admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:53.112680 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109791d6-a982-4967-9e97-f55f58031199 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:53.113140 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee0818d-0665-461c-bda5-5c0c451b634e admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:53.113751 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee0818d-0665-461c-bda5-5c0c451b634e admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:53.115584 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109791d6-a982-4967-9e97-f55f58031199 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:53.116779 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109791d6-a982-4967-9e97-f55f58031199 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:53.122587 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109791d6-a982-4967-9e97-f55f58031199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:53.123319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee0818d-0665-461c-bda5-5c0c451b634e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:53.129635 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109791d6-a982-4967-9e97-f55f58031199 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:53.131728 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee0818d-0665-461c-bda5-5c0c451b634e admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:53.168375 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 924/1837] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:53.169617 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 914/1838] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:47:53.170208 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109791d6-a982-4967-9e97-f55f58031199 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:53.172195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee0818d-0665-461c-bda5-5c0c451b634e admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:53.214362 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109791d6-a982-4967-9e97-f55f58031199 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:53.218010 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee0818d-0665-461c-bda5-5c0c451b634e admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:53.233571 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5126759f-90f1-4c00-8918-42e52f904816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:53.236656 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd090f-ae2a-411c-8705-56cd996da64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:53.237338 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126759f-90f1-4c00-8918-42e52f904816 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:53.239204 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126759f-90f1-4c00-8918-42e52f904816 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:53.239881 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd090f-ae2a-411c-8705-56cd996da64f admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:53.241789 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126759f-90f1-4c00-8918-42e52f904816 admin admin] PATH_INFO: `/v3/projects/624fd3f243d9417198334938e17b6483/users/cf01e67fe4af4aada544c6505ee4e3b9/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:53.242475 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd090f-ae2a-411c-8705-56cd996da64f admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:53.243823 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd090f-ae2a-411c-8705-56cd996da64f admin admin] PATH_INFO: `/v3/projects/519c39739b7748adb8939fcfeaef57c7/users/4a694b25307b4b4fb0a0efcb31b733f0/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:53.252322 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5126759f-90f1-4c00-8918-42e52f904816 admin admin] RBAC: Authorizing `identity:create_grant(project_id=624fd3f243d9417198334938e17b6483, user_id=cf01e67fe4af4aada544c6505ee4e3b9, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:53.258508 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5126759f-90f1-4c00-8918-42e52f904816 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:53.326068 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd090f-ae2a-411c-8705-56cd996da64f admin admin] RBAC: Authorizing `identity:create_grant(project_id=519c39739b7748adb8939fcfeaef57c7, user_id=4a694b25307b4b4fb0a0efcb31b733f0, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:53.327942 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 925/1839] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:47:53 2022] PUT /identity/v3/projects/624fd3f243d9417198334938e17b6483/users/cf01e67fe4af4aada544c6505ee4e3b9/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:53.333861 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd090f-ae2a-411c-8705-56cd996da64f admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:53.341867 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0eeb39-f672-4e80-93ee-593a511ab9b2 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:53.342468 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0eeb39-f672-4e80-93ee-593a511ab9b2 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:53.343038 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0eeb39-f672-4e80-93ee-593a511ab9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:53.398077 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 915/1840] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:47:53 2022] PUT /identity/v3/projects/519c39739b7748adb8939fcfeaef57c7/users/4a694b25307b4b4fb0a0efcb31b733f0/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:53.404581 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013194a-0f0c-44ee-9a78-0a6dab80cb57 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:53.405741 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013194a-0f0c-44ee-9a78-0a6dab80cb57 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:53.439542 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013194a-0f0c-44ee-9a78-0a6dab80cb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:53.501091 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-7f0eeb39-f672-4e80-93ee-593a511ab9b2 None None] MFA Rules not processed for user `cf01e67fe4af4aada544c6505ee4e3b9`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:53.504100 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-7f0eeb39-f672-4e80-93ee-593a511ab9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:53.538381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-b013194a-0f0c-44ee-9a78-0a6dab80cb57 None None] MFA Rules not processed for user `4a694b25307b4b4fb0a0efcb31b733f0`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:53.543820 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-b013194a-0f0c-44ee-9a78-0a6dab80cb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:53.670239 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:53.777025 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 926/1841] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:53 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 436 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:47:53.787349 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0eeb39-f672-4e80-93ee-593a511ab9b2 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:53.789504 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-7f0eeb39-f672-4e80-93ee-593a511ab9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:53.830538 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 916/1842] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 424 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:53.855193 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b013194a-0f0c-44ee-9a78-0a6dab80cb57 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:53.939452 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0eeb39-f672-4e80-93ee-593a511ab9b2 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:53.945147 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f00d7-fcb4-4e7d-a86f-d6e7595fa7a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:53.951042 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f00d7-fcb4-4e7d-a86f-d6e7595fa7a0 service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:53.951386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f00d7-fcb4-4e7d-a86f-d6e7595fa7a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:53.951618 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f00d7-fcb4-4e7d-a86f-d6e7595fa7a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:53.976083 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f00d7-fcb4-4e7d-a86f-d6e7595fa7a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:53.977924 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f00d7-fcb4-4e7d-a86f-d6e7595fa7a0 service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:53.998801 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 927/1843] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:53 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:54.026026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b013194a-0f0c-44ee-9a78-0a6dab80cb57 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:54.040614 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c67caf-7535-4a1f-81af-40d438b7e717 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:54.044757 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c67caf-7535-4a1f-81af-40d438b7e717 service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:54.048138 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c67caf-7535-4a1f-81af-40d438b7e717 service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:54.049143 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c67caf-7535-4a1f-81af-40d438b7e717 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:54.070118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c7224-d6dc-46f9-89d9-fb7db4396f6b None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:54.070620 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c7224-d6dc-46f9-89d9-fb7db4396f6b None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:54.071350 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c7224-d6dc-46f9-89d9-fb7db4396f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:54.095170 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c67caf-7535-4a1f-81af-40d438b7e717 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:54.099003 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c67caf-7535-4a1f-81af-40d438b7e717 service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:54.149821 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 917/1844] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:54.153469 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-6d3c7224-d6dc-46f9-89d9-fb7db4396f6b None None] MFA Rules not processed for user `3965cfcfffc348a98307954f2af92b5b`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:54.154865 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-6d3c7224-d6dc-46f9-89d9-fb7db4396f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:54.230503 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c67caf-7535-4a1f-81af-40d438b7e717 service cinder] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:54.240672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c67caf-7535-4a1f-81af-40d438b7e717 service cinder] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:54.254039 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fce9a3-b9c9-4d15-ad9a-8715febcbfec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:54.254711 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fce9a3-b9c9-4d15-ad9a-8715febcbfec service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:54.255151 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fce9a3-b9c9-4d15-ad9a-8715febcbfec service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:54.255430 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fce9a3-b9c9-4d15-ad9a-8715febcbfec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:54.312361 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:54.408883 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fce9a3-b9c9-4d15-ad9a-8715febcbfec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:54.412444 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fce9a3-b9c9-4d15-ad9a-8715febcbfec service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:54.440816 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 918/1845] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 22:47:54.471554 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 928/1846] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:47:54 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:55.193502 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab715e23-ce16-4e38-a4b5-dc4b589ca04b None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:55.194982 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab715e23-ce16-4e38-a4b5-dc4b589ca04b None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:55.194982 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab715e23-ce16-4e38-a4b5-dc4b589ca04b None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:55.196746 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 919/1847] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 10 22:47:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:47:55.274089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a4aad-63ce-4762-8e8b-c9c30fb57eca None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:55.274660 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a4aad-63ce-4762-8e8b-c9c30fb57eca None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:55.274660 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a4aad-63ce-4762-8e8b-c9c30fb57eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:55.346351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-280a4aad-63ce-4762-8e8b-c9c30fb57eca None None] MFA Rules not processed for user `b22838a8ad3949e59cdbaaa1801cd07f`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:55.349155 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-280a4aad-63ce-4762-8e8b-c9c30fb57eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:55.618802 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:55.651211 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 929/1848] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 10 22:47:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:56.891436 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab715e23-ce16-4e38-a4b5-dc4b589ca04b None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:56.893395 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-ab715e23-ce16-4e38-a4b5-dc4b589ca04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:56.944548 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab715e23-ce16-4e38-a4b5-dc4b589ca04b None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:56.954282 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a519c-8829-4d2c-82ee-07502d809733 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:56.957784 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a519c-8829-4d2c-82ee-07502d809733 service nova] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:56.958267 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a519c-8829-4d2c-82ee-07502d809733 service nova] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:56.958714 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a519c-8829-4d2c-82ee-07502d809733 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:56.999947 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a519c-8829-4d2c-82ee-07502d809733 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:57.002113 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a519c-8829-4d2c-82ee-07502d809733 service nova] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:57.059895 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 920/1849] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:57.897585 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680cb6e-29e5-473d-b875-89a13217ef60 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:57.898315 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680cb6e-29e5-473d-b875-89a13217ef60 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:57.899432 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680cb6e-29e5-473d-b875-89a13217ef60 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:57.901415 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 930/1850] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 10 22:47:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:47:57.925249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b8cee-dd24-460e-a677-a28b533a6f43 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:57.925756 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b8cee-dd24-460e-a677-a28b533a6f43 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:57.926068 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b8cee-dd24-460e-a677-a28b533a6f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:57.962523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-979b8cee-dd24-460e-a677-a28b533a6f43 None None] MFA Rules not processed for user `9f46dee0f73e49eab54c11556e3b86bf`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:57.966691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-979b8cee-dd24-460e-a677-a28b533a6f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:58.005424 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a1543-5fad-4497-b2b2-19911e586e52 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:58.006103 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a1543-5fad-4497-b2b2-19911e586e52 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:58.006954 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a1543-5fad-4497-b2b2-19911e586e52 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:58.009235 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 931/1851] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 10 22:47:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:47:58.062211 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 921/1852] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 10 22:47:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:58.090714 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a1543-5fad-4497-b2b2-19911e586e52 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:58.090714 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-834a1543-5fad-4497-b2b2-19911e586e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:58.095239 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6741d-c159-4ea9-859f-d9a72740b018 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:58.095883 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6741d-c159-4ea9-859f-d9a72740b018 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:58.095991 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6741d-c159-4ea9-859f-d9a72740b018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:58.109201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a1543-5fad-4497-b2b2-19911e586e52 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:58.114352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baacd83b-a6fd-42a5-8e5d-96d103002a0d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f46dee0f73e49eab54c11556e3b86bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:58.114952 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baacd83b-a6fd-42a5-8e5d-96d103002a0d service placement] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:58.115417 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baacd83b-a6fd-42a5-8e5d-96d103002a0d service placement] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:58.115906 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baacd83b-a6fd-42a5-8e5d-96d103002a0d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:58.119944 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-c9d6741d-c159-4ea9-859f-d9a72740b018 None None] MFA Rules not processed for user `b22838a8ad3949e59cdbaaa1801cd07f`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:58.121511 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-c9d6741d-c159-4ea9-859f-d9a72740b018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:58.127272 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baacd83b-a6fd-42a5-8e5d-96d103002a0d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:58.134020 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 922/1853] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 10 22:47:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:58.135191 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baacd83b-a6fd-42a5-8e5d-96d103002a0d service placement] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:58.146715 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 932/1854] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:58.151981 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d6741d-c159-4ea9-859f-d9a72740b018 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:58.164756 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d6741d-c159-4ea9-859f-d9a72740b018 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:58.170594 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd855003-0b43-4b11-9770-a6bb17ad04ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:58.171294 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd855003-0b43-4b11-9770-a6bb17ad04ea service nova] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:58.171645 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd855003-0b43-4b11-9770-a6bb17ad04ea service nova] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:58.171994 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd855003-0b43-4b11-9770-a6bb17ad04ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:58.182482 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd855003-0b43-4b11-9770-a6bb17ad04ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:58.184172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd855003-0b43-4b11-9770-a6bb17ad04ea service nova] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:58.195550 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 923/1855] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:58.937920 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e75c9d-ae86-428a-93f5-653ffff8a7b4 None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:58.938469 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e75c9d-ae86-428a-93f5-653ffff8a7b4 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:58.938469 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e75c9d-ae86-428a-93f5-653ffff8a7b4 None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:58.939819 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 933/1856] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 10 22:47:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:47:58.981968 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b3ed1-8aad-4097-9cb9-74d2ffee0510 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:58.983034 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b3ed1-8aad-4097-9cb9-74d2ffee0510 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:58.983800 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b3ed1-8aad-4097-9cb9-74d2ffee0510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:59.005330 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-198b3ed1-8aad-4097-9cb9-74d2ffee0510 None None] MFA Rules not processed for user `9f46dee0f73e49eab54c11556e3b86bf`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:47:59.006765 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-198b3ed1-8aad-4097-9cb9-74d2ffee0510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:59.015802 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 924/1857] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 10 22:47:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:47:59.031404 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e75c9d-ae86-428a-93f5-653ffff8a7b4 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:59.046379 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e75c9d-ae86-428a-93f5-653ffff8a7b4 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:59.052570 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886e3c0-0ac0-40a4-89f1-6cfa7923cef8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f46dee0f73e49eab54c11556e3b86bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:59.054048 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886e3c0-0ac0-40a4-89f1-6cfa7923cef8 service placement] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:59.054478 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886e3c0-0ac0-40a4-89f1-6cfa7923cef8 service placement] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:59.054952 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886e3c0-0ac0-40a4-89f1-6cfa7923cef8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:59.068438 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886e3c0-0ac0-40a4-89f1-6cfa7923cef8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:59.070753 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886e3c0-0ac0-40a4-89f1-6cfa7923cef8 service placement] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:59.077981 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b3ed1-8aad-4097-9cb9-74d2ffee0510 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:59.085849 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 934/1858] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:47:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:47:59.189093 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b3ed1-8aad-4097-9cb9-74d2ffee0510 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:59.199064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4828a73-2a91-4f7a-9107-ee940b05c4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:59.200532 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4828a73-2a91-4f7a-9107-ee940b05c4a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:59.201026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4828a73-2a91-4f7a-9107-ee940b05c4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:59.201565 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4828a73-2a91-4f7a-9107-ee940b05c4a1 admin admin] PATH_INFO: `/v3/users/b874dd1d3675426ea36bc55e7c660c85` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:59.203057 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4828a73-2a91-4f7a-9107-ee940b05c4a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b874dd1d3675426ea36bc55e7c660c85)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:59.205106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4828a73-2a91-4f7a-9107-ee940b05c4a1 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:59.230375 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:47:59.256927 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f4828a73-2a91-4f7a-9107-ee940b05c4a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b874dd1d3675426ea36bc55e7c660c85'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:59.263736 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f4828a73-2a91-4f7a-9107-ee940b05c4a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b874dd1d3675426ea36bc55e7c660c85'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:59.275469 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f4828a73-2a91-4f7a-9107-ee940b05c4a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b874dd1d3675426ea36bc55e7c660c85'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:59.276166 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f4828a73-2a91-4f7a-9107-ee940b05c4a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b874dd1d3675426ea36bc55e7c660c85'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:59.276784 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f4828a73-2a91-4f7a-9107-ee940b05c4a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b874dd1d3675426ea36bc55e7c660c85'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:59.289794 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 925/1859] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:47:59 2022] DELETE /identity/v3/users/b874dd1d3675426ea36bc55e7c660c85 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:59.464401 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886e3c0-0ac0-40a4-89f1-6cfa7923cef8 service placement] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:59.479294 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886e3c0-0ac0-40a4-89f1-6cfa7923cef8 service placement] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:59.550447 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad455c-d48d-497e-9a47-0754c42519cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:59.551090 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad455c-d48d-497e-9a47-0754c42519cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:59.551375 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad455c-d48d-497e-9a47-0754c42519cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:59.551649 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad455c-d48d-497e-9a47-0754c42519cc admin admin] PATH_INFO: `/v3/projects/a86160e5bd874af4b18cb5f2dbccf4b0` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:59.552789 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ad455c-d48d-497e-9a47-0754c42519cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a86160e5bd874af4b18cb5f2dbccf4b0)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:59.555871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ad455c-d48d-497e-9a47-0754c42519cc admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:59.605737 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-55ad455c-d48d-497e-9a47-0754c42519cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a86160e5bd874af4b18cb5f2dbccf4b0', 'request_id': 'req-55ad455c-d48d-497e-9a47-0754c42519cc'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:59.611289 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-55ad455c-d48d-497e-9a47-0754c42519cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a86160e5bd874af4b18cb5f2dbccf4b0', 'request_id': 'req-55ad455c-d48d-497e-9a47-0754c42519cc'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:59.619306 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-55ad455c-d48d-497e-9a47-0754c42519cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a86160e5bd874af4b18cb5f2dbccf4b0', 'request_id': 'req-55ad455c-d48d-497e-9a47-0754c42519cc'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:59.630807 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-55ad455c-d48d-497e-9a47-0754c42519cc admin admin] The token cache is being invalidate because project a86160e5bd874af4b18cb5f2dbccf4b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:47:59.631097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-55ad455c-d48d-497e-9a47-0754c42519cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:59.631810 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-55ad455c-d48d-497e-9a47-0754c42519cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:59.633317 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 935/1860] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:47:59 2022] DELETE /identity/v3/projects/a86160e5bd874af4b18cb5f2dbccf4b0 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:59.637302 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4828a73-2a91-4f7a-9107-ee940b05c4a1 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:59.638753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-f4828a73-2a91-4f7a-9107-ee940b05c4a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:47:59.651729 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4828a73-2a91-4f7a-9107-ee940b05c4a1 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:47:59.713836 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0963c28c-6c11-4aae-9692-ef90e3a80393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:47:59.714483 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963c28c-6c11-4aae-9692-ef90e3a80393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:47:59.714767 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963c28c-6c11-4aae-9692-ef90e3a80393 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:47:59.715043 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963c28c-6c11-4aae-9692-ef90e3a80393 admin admin] PATH_INFO: `/v3/users/3965cfcfffc348a98307954f2af92b5b` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:47:59.716170 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0963c28c-6c11-4aae-9692-ef90e3a80393 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3965cfcfffc348a98307954f2af92b5b)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:47:59.718145 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0963c28c-6c11-4aae-9692-ef90e3a80393 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:47:59.771842 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-0963c28c-6c11-4aae-9692-ef90e3a80393 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3965cfcfffc348a98307954f2af92b5b'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:59.780418 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-0963c28c-6c11-4aae-9692-ef90e3a80393 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3965cfcfffc348a98307954f2af92b5b'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:59.792789 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-0963c28c-6c11-4aae-9692-ef90e3a80393 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3965cfcfffc348a98307954f2af92b5b'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:59.793553 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-0963c28c-6c11-4aae-9692-ef90e3a80393 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3965cfcfffc348a98307954f2af92b5b'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:59.794175 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-0963c28c-6c11-4aae-9692-ef90e3a80393 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3965cfcfffc348a98307954f2af92b5b'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:47:59.808207 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 926/1861] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:47:59 2022] DELETE /identity/v3/users/3965cfcfffc348a98307954f2af92b5b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:47:59.986612 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad455c-d48d-497e-9a47-0754c42519cc admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:47:59.988087 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-55ad455c-d48d-497e-9a47-0754c42519cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:00.001541 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad455c-d48d-497e-9a47-0754c42519cc admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:00.065301 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d31f98-32dc-4227-8125-ddc23173fc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:00.066221 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d31f98-32dc-4227-8125-ddc23173fc65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:00.066624 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d31f98-32dc-4227-8125-ddc23173fc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:00.067023 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d31f98-32dc-4227-8125-ddc23173fc65 admin admin] PATH_INFO: `/v3/projects/2378d661540746018725207421ec14fd` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:00.068159 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d31f98-32dc-4227-8125-ddc23173fc65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2378d661540746018725207421ec14fd)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:00.070321 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d31f98-32dc-4227-8125-ddc23173fc65 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:00.076736 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:00.117899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-33d31f98-32dc-4227-8125-ddc23173fc65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2378d661540746018725207421ec14fd', 'request_id': 'req-33d31f98-32dc-4227-8125-ddc23173fc65'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:00.123723 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-33d31f98-32dc-4227-8125-ddc23173fc65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2378d661540746018725207421ec14fd', 'request_id': 'req-33d31f98-32dc-4227-8125-ddc23173fc65'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:00.132753 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-33d31f98-32dc-4227-8125-ddc23173fc65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2378d661540746018725207421ec14fd', 'request_id': 'req-33d31f98-32dc-4227-8125-ddc23173fc65'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:00.147599 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-33d31f98-32dc-4227-8125-ddc23173fc65 admin admin] The token cache is being invalidate because project 2378d661540746018725207421ec14fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:48:00.147898 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-33d31f98-32dc-4227-8125-ddc23173fc65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:00.148573 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-33d31f98-32dc-4227-8125-ddc23173fc65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:00.150095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 936/1862] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:47:59 2022] DELETE /identity/v3/projects/2378d661540746018725207421ec14fd => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:48:00.154776 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4875a4-0d1d-4857-a914-07e58e6d52a7 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:00.155736 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4875a4-0d1d-4857-a914-07e58e6d52a7 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:00.156022 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4875a4-0d1d-4857-a914-07e58e6d52a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:00.175692 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-6c4875a4-0d1d-4857-a914-07e58e6d52a7 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:00.176917 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-6c4875a4-0d1d-4857-a914-07e58e6d52a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:00.190691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:00.251004 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 927/1863] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:48:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:48:00.259055 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf09e2-486e-43aa-8fd3-0551520a3719 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:00.259342 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf09e2-486e-43aa-8fd3-0551520a3719 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:00.259616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf09e2-486e-43aa-8fd3-0551520a3719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:00.280381 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-4cbf09e2-486e-43aa-8fd3-0551520a3719 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:00.281624 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-4cbf09e2-486e-43aa-8fd3-0551520a3719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:00.290066 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 937/1864] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:48:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:48:00.294150 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4875a4-0d1d-4857-a914-07e58e6d52a7 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:00.304801 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4875a4-0d1d-4857-a914-07e58e6d52a7 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:00.309416 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf433fe5-0be6-4ea9-9325-2723188e65c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:00.310023 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf433fe5-0be6-4ea9-9325-2723188e65c1 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:00.310299 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf433fe5-0be6-4ea9-9325-2723188e65c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:00.310566 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf433fe5-0be6-4ea9-9325-2723188e65c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:00.311222 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf433fe5-0be6-4ea9-9325-2723188e65c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:48:00.311509 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf433fe5-0be6-4ea9-9325-2723188e65c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:00.313093 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf433fe5-0be6-4ea9-9325-2723188e65c1 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:00.326955 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 928/1865] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:48:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:48:00.331410 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf09e2-486e-43aa-8fd3-0551520a3719 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:00.341631 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf09e2-486e-43aa-8fd3-0551520a3719 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:00.346945 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130170d7-f5b1-4289-895a-52e693458ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:00.347633 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130170d7-f5b1-4289-895a-52e693458ebc admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:00.348033 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130170d7-f5b1-4289-895a-52e693458ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:00.348374 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130170d7-f5b1-4289-895a-52e693458ebc admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:00.349078 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130170d7-f5b1-4289-895a-52e693458ebc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:00.351411 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130170d7-f5b1-4289-895a-52e693458ebc admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:00.395789 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 938/1866] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:48:00 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:48:00.398045 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf433fe5-0be6-4ea9-9325-2723188e65c1 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:00.408805 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf433fe5-0be6-4ea9-9325-2723188e65c1 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:00.471264 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d21bd8-eef1-4df4-a66f-e754dcf572f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:00.471904 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d21bd8-eef1-4df4-a66f-e754dcf572f4 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:00.472182 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d21bd8-eef1-4df4-a66f-e754dcf572f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:00.472452 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d21bd8-eef1-4df4-a66f-e754dcf572f4 admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:00.473061 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d21bd8-eef1-4df4-a66f-e754dcf572f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:00.475072 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d21bd8-eef1-4df4-a66f-e754dcf572f4 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:00.496083 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 929/1867] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:48:00 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:48:00.500528 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130170d7-f5b1-4289-895a-52e693458ebc admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:00.511825 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130170d7-f5b1-4289-895a-52e693458ebc admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:00.516445 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc0a193-413d-48ba-970c-707030eb9c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:00.517043 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0a193-413d-48ba-970c-707030eb9c9b admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:00.517362 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0a193-413d-48ba-970c-707030eb9c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:00.517640 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0a193-413d-48ba-970c-707030eb9c9b admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:00.518283 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc0a193-413d-48ba-970c-707030eb9c9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:00.519891 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc0a193-413d-48ba-970c-707030eb9c9b admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:00.531408 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 939/1868] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:48:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:48:00.535490 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d21bd8-eef1-4df4-a66f-e754dcf572f4 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:00.546075 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d21bd8-eef1-4df4-a66f-e754dcf572f4 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:00.550721 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be780eff-0d9c-4c59-aaf4-e32362a38da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:00.551322 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be780eff-0d9c-4c59-aaf4-e32362a38da9 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:00.551601 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be780eff-0d9c-4c59-aaf4-e32362a38da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:00.551873 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be780eff-0d9c-4c59-aaf4-e32362a38da9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:00.552512 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be780eff-0d9c-4c59-aaf4-e32362a38da9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:00.554832 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be780eff-0d9c-4c59-aaf4-e32362a38da9 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:00.566611 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 930/1869] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:48:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:48:00.571765 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc0a193-413d-48ba-970c-707030eb9c9b admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:00.582690 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc0a193-413d-48ba-970c-707030eb9c9b admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:00.593186 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a6c3d-9c7c-4195-8bd7-285bb437552a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:00.593883 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a6c3d-9c7c-4195-8bd7-285bb437552a admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:00.594166 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a6c3d-9c7c-4195-8bd7-285bb437552a admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:00.594442 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a6c3d-9c7c-4195-8bd7-285bb437552a admin admin] PATH_INFO: `/v3/projects/1545b3fa31274b879c523897476eee09/users/c2d0bf2ba7274336bd201872eea4d2c8/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:00.609019 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a6c3d-9c7c-4195-8bd7-285bb437552a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1545b3fa31274b879c523897476eee09, user_id=c2d0bf2ba7274336bd201872eea4d2c8, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:00.611773 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a6c3d-9c7c-4195-8bd7-285bb437552a admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:00.625570 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 940/1870] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:48:00 2022] PUT /identity/v3/projects/1545b3fa31274b879c523897476eee09/users/c2d0bf2ba7274336bd201872eea4d2c8/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:48:00.637747 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f510e3-8c55-448c-a484-8cf516783138 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:00.638032 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f510e3-8c55-448c-a484-8cf516783138 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:00.638308 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f510e3-8c55-448c-a484-8cf516783138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:00.665606 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-c7f510e3-8c55-448c-a484-8cf516783138 None None] MFA Rules not processed for user `c2d0bf2ba7274336bd201872eea4d2c8`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:00.666808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-c7f510e3-8c55-448c-a484-8cf516783138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:00.730574 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 931/1871] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:48:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:48:00.735300 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a6c3d-9c7c-4195-8bd7-285bb437552a admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:00.804620 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a6c3d-9c7c-4195-8bd7-285bb437552a admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:00.809420 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f474548-06d9-42bd-9b4e-eab57dd7d29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:00.810030 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f474548-06d9-42bd-9b4e-eab57dd7d29f admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:00.810313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f474548-06d9-42bd-9b4e-eab57dd7d29f admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:00.810594 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f474548-06d9-42bd-9b4e-eab57dd7d29f admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:00.811226 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f474548-06d9-42bd-9b4e-eab57dd7d29f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:00.813162 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f474548-06d9-42bd-9b4e-eab57dd7d29f admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:00.845619 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 941/1872] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:48:00 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:48:00.846898 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f510e3-8c55-448c-a484-8cf516783138 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:00.858305 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f510e3-8c55-448c-a484-8cf516783138 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:00.872190 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:00.929601 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4414226b-ce60-4872-a18c-fefe300b47ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:00.930377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414226b-ce60-4872-a18c-fefe300b47ca admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:00.930772 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414226b-ce60-4872-a18c-fefe300b47ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:00.930947 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414226b-ce60-4872-a18c-fefe300b47ca admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:00.931576 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4414226b-ce60-4872-a18c-fefe300b47ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:00.933531 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4414226b-ce60-4872-a18c-fefe300b47ca admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:00.956166 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 932/1873] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:48:00 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:48:00.961219 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f474548-06d9-42bd-9b4e-eab57dd7d29f admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:00.971523 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f474548-06d9-42bd-9b4e-eab57dd7d29f admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:00.976496 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebf6d23-cabe-4205-ab8a-319b6815045d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:00.977097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf6d23-cabe-4205-ab8a-319b6815045d admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:00.977440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf6d23-cabe-4205-ab8a-319b6815045d admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:00.977721 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf6d23-cabe-4205-ab8a-319b6815045d admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:00.978364 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebf6d23-cabe-4205-ab8a-319b6815045d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:00.979960 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebf6d23-cabe-4205-ab8a-319b6815045d admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:00.990372 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 942/1874] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:48:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:48:00.994907 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4414226b-ce60-4872-a18c-fefe300b47ca admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:01.004774 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4414226b-ce60-4872-a18c-fefe300b47ca admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:01.010131 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01233dcc-5fc6-474c-afb8-312310a992b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:01.010777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01233dcc-5fc6-474c-afb8-312310a992b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:01.011217 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01233dcc-5fc6-474c-afb8-312310a992b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:01.011464 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01233dcc-5fc6-474c-afb8-312310a992b6 admin admin] PATH_INFO: `/v3/projects/c1821fb9d4fe4610a61e465663b00261/users/b57da6d2359044dc8ae0be1ec2fbe7e8/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:01.026696 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01233dcc-5fc6-474c-afb8-312310a992b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1821fb9d4fe4610a61e465663b00261, user_id=b57da6d2359044dc8ae0be1ec2fbe7e8, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:01.029673 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01233dcc-5fc6-474c-afb8-312310a992b6 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:01.042804 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 933/1875] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:48:00 2022] PUT /identity/v3/projects/c1821fb9d4fe4610a61e465663b00261/users/b57da6d2359044dc8ae0be1ec2fbe7e8/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:48:01.046724 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebf6d23-cabe-4205-ab8a-319b6815045d admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:01.053839 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:01.058452 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebf6d23-cabe-4205-ab8a-319b6815045d admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:01.116469 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effd8d47-23fe-40fd-a702-20dc940dbf2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:01.117081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd8d47-23fe-40fd-a702-20dc940dbf2b admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:01.117385 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd8d47-23fe-40fd-a702-20dc940dbf2b admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:01.117660 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd8d47-23fe-40fd-a702-20dc940dbf2b admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:01.118304 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effd8d47-23fe-40fd-a702-20dc940dbf2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:01.119905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effd8d47-23fe-40fd-a702-20dc940dbf2b admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:01.129707 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 943/1876] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:48:01.135021 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01233dcc-5fc6-474c-afb8-312310a992b6 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:01.145308 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01233dcc-5fc6-474c-afb8-312310a992b6 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:01.150661 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e2e996-72a5-4818-94ae-150323df8237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:01.151325 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2e996-72a5-4818-94ae-150323df8237 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:01.151605 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2e996-72a5-4818-94ae-150323df8237 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:01.151880 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2e996-72a5-4818-94ae-150323df8237 admin admin] PATH_INFO: `/v3/projects/c1821fb9d4fe4610a61e465663b00261/users/b57da6d2359044dc8ae0be1ec2fbe7e8/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:01.153807 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e2e996-72a5-4818-94ae-150323df8237 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1821fb9d4fe4610a61e465663b00261, user_id=b57da6d2359044dc8ae0be1ec2fbe7e8, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:01.156465 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e2e996-72a5-4818-94ae-150323df8237 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:01.167481 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 934/1877] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:48:01 2022] PUT /identity/v3/projects/c1821fb9d4fe4610a61e465663b00261/users/b57da6d2359044dc8ae0be1ec2fbe7e8/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:48:01.180312 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce19cdc-f787-4c01-96dd-ae0b215cdf2d None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:01.180551 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce19cdc-f787-4c01-96dd-ae0b215cdf2d None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:01.181375 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce19cdc-f787-4c01-96dd-ae0b215cdf2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:01.214836 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-5ce19cdc-f787-4c01-96dd-ae0b215cdf2d None None] MFA Rules not processed for user `b57da6d2359044dc8ae0be1ec2fbe7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:01.216050 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-5ce19cdc-f787-4c01-96dd-ae0b215cdf2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:01.282889 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 944/1878] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:48:01 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:48:01.294197 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e2e996-72a5-4818-94ae-150323df8237 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:01.295898 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-f1e2e996-72a5-4818-94ae-150323df8237 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:01.353030 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e2e996-72a5-4818-94ae-150323df8237 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:01.356657 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b11e90-4597-4f21-8fc5-403f92875e35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:01.357267 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b11e90-4597-4f21-8fc5-403f92875e35 service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:01.357574 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b11e90-4597-4f21-8fc5-403f92875e35 service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:01.357845 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b11e90-4597-4f21-8fc5-403f92875e35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:01.367438 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b11e90-4597-4f21-8fc5-403f92875e35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:01.368958 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b11e90-4597-4f21-8fc5-403f92875e35 service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:01.379971 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 935/1879] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:48:01 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:48:01.477021 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce19cdc-f787-4c01-96dd-ae0b215cdf2d None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:01.487780 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce19cdc-f787-4c01-96dd-ae0b215cdf2d None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:01.494671 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d375ef2c-a037-4997-9e2f-d25fb4e6e773 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:01.495370 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375ef2c-a037-4997-9e2f-d25fb4e6e773 service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:01.495727 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375ef2c-a037-4997-9e2f-d25fb4e6e773 service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:01.496067 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375ef2c-a037-4997-9e2f-d25fb4e6e773 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:01.576303 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d375ef2c-a037-4997-9e2f-d25fb4e6e773 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:01.577838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d375ef2c-a037-4997-9e2f-d25fb4e6e773 service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:01.588638 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 945/1880] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:48:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:48:03.774966 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b11e90-4597-4f21-8fc5-403f92875e35 service cinder] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:03.777062 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-95b11e90-4597-4f21-8fc5-403f92875e35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:03.880445 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b11e90-4597-4f21-8fc5-403f92875e35 service cinder] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:03.885849 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c4f1e-6bf4-4e19-a8ee-f617852460fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:03.886481 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c4f1e-6bf4-4e19-a8ee-f617852460fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:03.886757 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c4f1e-6bf4-4e19-a8ee-f617852460fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:03.887058 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c4f1e-6bf4-4e19-a8ee-f617852460fc admin admin] PATH_INFO: `/v3/users/7157eb675616440294a4bced8239e173` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:03.888171 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83c4f1e-6bf4-4e19-a8ee-f617852460fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7157eb675616440294a4bced8239e173)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:03.890098 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83c4f1e-6bf4-4e19-a8ee-f617852460fc admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:03.914123 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:04.020912 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-d83c4f1e-6bf4-4e19-a8ee-f617852460fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7157eb675616440294a4bced8239e173'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:04.055309 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-d83c4f1e-6bf4-4e19-a8ee-f617852460fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7157eb675616440294a4bced8239e173'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:04.086985 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-d83c4f1e-6bf4-4e19-a8ee-f617852460fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7157eb675616440294a4bced8239e173'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:04.087793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-d83c4f1e-6bf4-4e19-a8ee-f617852460fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7157eb675616440294a4bced8239e173'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:04.088811 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-d83c4f1e-6bf4-4e19-a8ee-f617852460fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7157eb675616440294a4bced8239e173'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:04.124144 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 936/1881] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:48:03 2022] DELETE /identity/v3/users/7157eb675616440294a4bced8239e173 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:48:04.131981 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096c6c1-76c9-439c-8825-9d857dca1106 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:04.131981 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096c6c1-76c9-439c-8825-9d857dca1106 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:04.131981 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096c6c1-76c9-439c-8825-9d857dca1106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:04.153651 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c4f1e-6bf4-4e19-a8ee-f617852460fc admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:04.155293 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-d83c4f1e-6bf4-4e19-a8ee-f617852460fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:04.168615 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-4096c6c1-76c9-439c-8825-9d857dca1106 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:04.170036 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-4096c6c1-76c9-439c-8825-9d857dca1106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:04.238369 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:04.307109 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c4f1e-6bf4-4e19-a8ee-f617852460fc admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:04.321223 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9bb09-4a49-4764-9966-0a92005b6daf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:04.322348 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9bb09-4a49-4764-9966-0a92005b6daf service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:04.322701 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9bb09-4a49-4764-9966-0a92005b6daf service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:04.323189 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9bb09-4a49-4764-9966-0a92005b6daf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:04.326035 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-f2b9bb09-4a49-4764-9966-0a92005b6daf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:04.351697 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 946/1882] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:48:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:48:04.362096 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096c6c1-76c9-439c-8825-9d857dca1106 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:04.363877 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-4096c6c1-76c9-439c-8825-9d857dca1106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:04.502192 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096c6c1-76c9-439c-8825-9d857dca1106 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:04.520026 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7839e5-b6ba-49d6-9494-fe2421e0f7cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:04.520727 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7839e5-b6ba-49d6-9494-fe2421e0f7cb service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:04.521018 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7839e5-b6ba-49d6-9494-fe2421e0f7cb service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:04.521320 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7839e5-b6ba-49d6-9494-fe2421e0f7cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:04.531424 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b9bb09-4a49-4764-9966-0a92005b6daf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:04.533626 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b9bb09-4a49-4764-9966-0a92005b6daf service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:04.534353 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7839e5-b6ba-49d6-9494-fe2421e0f7cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:04.535829 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7839e5-b6ba-49d6-9494-fe2421e0f7cb service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:04.551689 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 947/1883] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:48:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 22:48:04.562622 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 937/1884] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:48:04 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 22:48:05.100159 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7839e5-b6ba-49d6-9494-fe2421e0f7cb service neutron] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:05.102467 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-8f7839e5-b6ba-49d6-9494-fe2421e0f7cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:05.134777 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7839e5-b6ba-49d6-9494-fe2421e0f7cb service neutron] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:05.139959 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841bbb33-8ed4-447c-a7d9-bc41097bb2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:05.140564 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bbb33-8ed4-447c-a7d9-bc41097bb2e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:05.140800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bbb33-8ed4-447c-a7d9-bc41097bb2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:05.141079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bbb33-8ed4-447c-a7d9-bc41097bb2e8 admin admin] PATH_INFO: `/v3/projects/06a020c9f2f24235b4d728574d5a8127` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:05.142283 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841bbb33-8ed4-447c-a7d9-bc41097bb2e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06a020c9f2f24235b4d728574d5a8127)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:05.144334 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841bbb33-8ed4-447c-a7d9-bc41097bb2e8 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:05.214601 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-841bbb33-8ed4-447c-a7d9-bc41097bb2e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06a020c9f2f24235b4d728574d5a8127', 'request_id': 'req-841bbb33-8ed4-447c-a7d9-bc41097bb2e8'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:05.220606 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-841bbb33-8ed4-447c-a7d9-bc41097bb2e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06a020c9f2f24235b4d728574d5a8127', 'request_id': 'req-841bbb33-8ed4-447c-a7d9-bc41097bb2e8'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:05.232796 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-841bbb33-8ed4-447c-a7d9-bc41097bb2e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06a020c9f2f24235b4d728574d5a8127', 'request_id': 'req-841bbb33-8ed4-447c-a7d9-bc41097bb2e8'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:05.248425 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-841bbb33-8ed4-447c-a7d9-bc41097bb2e8 admin admin] The token cache is being invalidate because project 06a020c9f2f24235b4d728574d5a8127 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:48:05.248721 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-841bbb33-8ed4-447c-a7d9-bc41097bb2e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:05.249473 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-841bbb33-8ed4-447c-a7d9-bc41097bb2e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:05.255003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 948/1885] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:48:05 2022] DELETE /identity/v3/projects/06a020c9f2f24235b4d728574d5a8127 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:48:05.255260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9bb09-4a49-4764-9966-0a92005b6daf service cinder] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:05.276600 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9bb09-4a49-4764-9966-0a92005b6daf service cinder] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:05.349437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:05.373195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2805e15-74b9-4bce-b9be-dfd7c59f2077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:05.373888 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2805e15-74b9-4bce-b9be-dfd7c59f2077 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:05.374159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2805e15-74b9-4bce-b9be-dfd7c59f2077 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:05.374426 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2805e15-74b9-4bce-b9be-dfd7c59f2077 admin admin] PATH_INFO: `/v3/users/cf01e67fe4af4aada544c6505ee4e3b9` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:05.375589 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2805e15-74b9-4bce-b9be-dfd7c59f2077 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf01e67fe4af4aada544c6505ee4e3b9)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:05.380587 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2805e15-74b9-4bce-b9be-dfd7c59f2077 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:05.456661 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f2805e15-74b9-4bce-b9be-dfd7c59f2077 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf01e67fe4af4aada544c6505ee4e3b9'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:05.467503 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f2805e15-74b9-4bce-b9be-dfd7c59f2077 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf01e67fe4af4aada544c6505ee4e3b9'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:05.482939 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f2805e15-74b9-4bce-b9be-dfd7c59f2077 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf01e67fe4af4aada544c6505ee4e3b9'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:05.483775 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f2805e15-74b9-4bce-b9be-dfd7c59f2077 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf01e67fe4af4aada544c6505ee4e3b9'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:05.484470 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f2805e15-74b9-4bce-b9be-dfd7c59f2077 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf01e67fe4af4aada544c6505ee4e3b9'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:05.525838 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 938/1886] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:48:05 2022] DELETE /identity/v3/users/cf01e67fe4af4aada544c6505ee4e3b9 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:48:05.869124 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841bbb33-8ed4-447c-a7d9-bc41097bb2e8 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:05.870701 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-841bbb33-8ed4-447c-a7d9-bc41097bb2e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:05.886201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841bbb33-8ed4-447c-a7d9-bc41097bb2e8 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:05.937673 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:05.950244 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974a7129-4c84-46f2-9977-675640528e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:05.950931 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a7129-4c84-46f2-9977-675640528e16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:05.951392 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a7129-4c84-46f2-9977-675640528e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:05.951747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a7129-4c84-46f2-9977-675640528e16 admin admin] PATH_INFO: `/v3/projects/624fd3f243d9417198334938e17b6483` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:05.953020 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974a7129-4c84-46f2-9977-675640528e16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=624fd3f243d9417198334938e17b6483)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:05.955259 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974a7129-4c84-46f2-9977-675640528e16 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:06.002873 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-974a7129-4c84-46f2-9977-675640528e16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '624fd3f243d9417198334938e17b6483', 'request_id': 'req-974a7129-4c84-46f2-9977-675640528e16'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:06.009596 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-974a7129-4c84-46f2-9977-675640528e16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '624fd3f243d9417198334938e17b6483', 'request_id': 'req-974a7129-4c84-46f2-9977-675640528e16'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:06.018640 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-974a7129-4c84-46f2-9977-675640528e16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '624fd3f243d9417198334938e17b6483', 'request_id': 'req-974a7129-4c84-46f2-9977-675640528e16'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:06.029913 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-974a7129-4c84-46f2-9977-675640528e16 admin admin] The token cache is being invalidate because project 624fd3f243d9417198334938e17b6483 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:48:06.030202 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-974a7129-4c84-46f2-9977-675640528e16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:06.030885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-974a7129-4c84-46f2-9977-675640528e16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:06.032578 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 949/1887] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:48:05 2022] DELETE /identity/v3/projects/624fd3f243d9417198334938e17b6483 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:48:06.037497 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18caf85-77d0-4004-8003-25d1218320a8 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:06.038018 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18caf85-77d0-4004-8003-25d1218320a8 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:06.038413 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18caf85-77d0-4004-8003-25d1218320a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:06.061141 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-c18caf85-77d0-4004-8003-25d1218320a8 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:06.062326 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-c18caf85-77d0-4004-8003-25d1218320a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:06.121593 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 939/1888] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:48:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:48:06.129248 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29a495-8ef8-460e-8f0f-7adc583266d7 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:06.129582 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29a495-8ef8-460e-8f0f-7adc583266d7 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:06.129853 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29a495-8ef8-460e-8f0f-7adc583266d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:06.149873 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-5f29a495-8ef8-460e-8f0f-7adc583266d7 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:06.151094 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-5f29a495-8ef8-460e-8f0f-7adc583266d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:06.159068 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 950/1889] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:48:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:48:06.163910 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18caf85-77d0-4004-8003-25d1218320a8 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:06.173601 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18caf85-77d0-4004-8003-25d1218320a8 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:06.178809 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80a13b-95ef-4ecc-b181-9ab14ea3b3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:06.179404 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80a13b-95ef-4ecc-b181-9ab14ea3b3cd admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:06.179683 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80a13b-95ef-4ecc-b181-9ab14ea3b3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:06.179951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80a13b-95ef-4ecc-b181-9ab14ea3b3cd admin admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:06.180616 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc80a13b-95ef-4ecc-b181-9ab14ea3b3cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:48:06.180910 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc80a13b-95ef-4ecc-b181-9ab14ea3b3cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:06.182541 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc80a13b-95ef-4ecc-b181-9ab14ea3b3cd admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:06.193937 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 940/1890] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:48:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:48:06.198142 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29a495-8ef8-460e-8f0f-7adc583266d7 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:06.207390 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29a495-8ef8-460e-8f0f-7adc583266d7 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:06.212047 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d04061-4290-4d05-81c3-da4fd4ee0529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:06.212648 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d04061-4290-4d05-81c3-da4fd4ee0529 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:06.212927 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d04061-4290-4d05-81c3-da4fd4ee0529 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:06.213197 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d04061-4290-4d05-81c3-da4fd4ee0529 admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:06.213860 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d04061-4290-4d05-81c3-da4fd4ee0529 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:06.215767 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d04061-4290-4d05-81c3-da4fd4ee0529 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:06.244000 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 951/1891] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:48:06 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:48:06.248077 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80a13b-95ef-4ecc-b181-9ab14ea3b3cd admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:06.258633 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80a13b-95ef-4ecc-b181-9ab14ea3b3cd admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:06.301828 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:06.316471 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c46df23-8d07-4c67-ad61-f91be4423dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:06.317010 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46df23-8d07-4c67-ad61-f91be4423dfb admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:06.317308 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46df23-8d07-4c67-ad61-f91be4423dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:06.317584 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46df23-8d07-4c67-ad61-f91be4423dfb admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:06.318211 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c46df23-8d07-4c67-ad61-f91be4423dfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:06.320285 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c46df23-8d07-4c67-ad61-f91be4423dfb admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:06.343502 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 941/1892] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:48:06 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:48:06.347661 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d04061-4290-4d05-81c3-da4fd4ee0529 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:06.361378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d04061-4290-4d05-81c3-da4fd4ee0529 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:06.367111 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d645d18-5eb0-4587-a868-2181acab2d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:06.367725 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d645d18-5eb0-4587-a868-2181acab2d64 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:06.368014 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d645d18-5eb0-4587-a868-2181acab2d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:06.368283 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d645d18-5eb0-4587-a868-2181acab2d64 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:06.368925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d645d18-5eb0-4587-a868-2181acab2d64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:06.370560 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d645d18-5eb0-4587-a868-2181acab2d64 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:06.380526 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 952/1893] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:48:06.384600 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c46df23-8d07-4c67-ad61-f91be4423dfb admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:06.394604 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c46df23-8d07-4c67-ad61-f91be4423dfb admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:06.399217 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381fef9-fcf6-446d-8732-18853013f6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:06.399811 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381fef9-fcf6-446d-8732-18853013f6c6 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:06.400095 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381fef9-fcf6-446d-8732-18853013f6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:06.400368 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381fef9-fcf6-446d-8732-18853013f6c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:06.401004 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8381fef9-fcf6-446d-8732-18853013f6c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:06.402677 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8381fef9-fcf6-446d-8732-18853013f6c6 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:06.414191 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 942/1894] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:48:06.421689 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d645d18-5eb0-4587-a868-2181acab2d64 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:06.438236 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d645d18-5eb0-4587-a868-2181acab2d64 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:06.443319 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b1a35-c3c5-4625-8cec-c1c169188c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:06.444041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b1a35-c3c5-4625-8cec-c1c169188c06 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:06.444275 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b1a35-c3c5-4625-8cec-c1c169188c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:06.444554 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b1a35-c3c5-4625-8cec-c1c169188c06 admin admin] PATH_INFO: `/v3/projects/ca7c14f24ef743649ef07406dc897237/users/6cf05194be26416eb5b9f03103603b04/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:06.460304 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5b1a35-c3c5-4625-8cec-c1c169188c06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca7c14f24ef743649ef07406dc897237, user_id=6cf05194be26416eb5b9f03103603b04, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:06.463001 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5b1a35-c3c5-4625-8cec-c1c169188c06 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:06.476376 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 953/1895] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:48:06 2022] PUT /identity/v3/projects/ca7c14f24ef743649ef07406dc897237/users/6cf05194be26416eb5b9f03103603b04/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:48:06.486618 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb3add-4fec-4754-8d38-5309f9a4ed85 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:06.486907 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb3add-4fec-4754-8d38-5309f9a4ed85 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:06.487182 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb3add-4fec-4754-8d38-5309f9a4ed85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:06.516764 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-99eb3add-4fec-4754-8d38-5309f9a4ed85 None None] MFA Rules not processed for user `6cf05194be26416eb5b9f03103603b04`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:06.518058 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-99eb3add-4fec-4754-8d38-5309f9a4ed85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:06.595693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 943/1896] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:48:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:48:06.600819 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b1a35-c3c5-4625-8cec-c1c169188c06 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:06.670205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b1a35-c3c5-4625-8cec-c1c169188c06 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:06.675725 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1a08f-ac3a-4420-a30a-ff17d2becf3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:06.676356 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1a08f-ac3a-4420-a30a-ff17d2becf3b admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:06.676665 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1a08f-ac3a-4420-a30a-ff17d2becf3b admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:06.676934 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1a08f-ac3a-4420-a30a-ff17d2becf3b admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:06.677589 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f1a08f-ac3a-4420-a30a-ff17d2becf3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:06.679831 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f1a08f-ac3a-4420-a30a-ff17d2becf3b admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:06.708381 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 954/1897] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:48:06 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:48:06.712971 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb3add-4fec-4754-8d38-5309f9a4ed85 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:06.724349 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb3add-4fec-4754-8d38-5309f9a4ed85 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:06.788349 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17536589-3f3e-452c-b4eb-76da42733069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:06.789098 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17536589-3f3e-452c-b4eb-76da42733069 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:06.789515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17536589-3f3e-452c-b4eb-76da42733069 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:06.789880 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17536589-3f3e-452c-b4eb-76da42733069 admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:06.790514 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17536589-3f3e-452c-b4eb-76da42733069 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:06.792473 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17536589-3f3e-452c-b4eb-76da42733069 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:06.815423 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 944/1898] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:48:06 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:48:06.819368 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1a08f-ac3a-4420-a30a-ff17d2becf3b admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:06.825007 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:06.831889 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1a08f-ac3a-4420-a30a-ff17d2becf3b admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:06.839588 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf794fd2-c9f8-4eed-afbb-1772bb55fe36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:06.840283 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf794fd2-c9f8-4eed-afbb-1772bb55fe36 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:06.840569 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf794fd2-c9f8-4eed-afbb-1772bb55fe36 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:06.840940 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf794fd2-c9f8-4eed-afbb-1772bb55fe36 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:06.841727 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf794fd2-c9f8-4eed-afbb-1772bb55fe36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:06.843583 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf794fd2-c9f8-4eed-afbb-1772bb55fe36 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:06.855583 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 955/1899] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:48:06.859571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17536589-3f3e-452c-b4eb-76da42733069 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:06.870179 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17536589-3f3e-452c-b4eb-76da42733069 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:06.876212 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d7ad8-01a3-4477-b499-98298218a97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:06.876859 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d7ad8-01a3-4477-b499-98298218a97c admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:06.877179 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d7ad8-01a3-4477-b499-98298218a97c admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:06.877481 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d7ad8-01a3-4477-b499-98298218a97c admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:06.878229 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7d7ad8-01a3-4477-b499-98298218a97c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:06.879829 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7d7ad8-01a3-4477-b499-98298218a97c admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:06.894791 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 945/1900] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 22:48:06.902657 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf794fd2-c9f8-4eed-afbb-1772bb55fe36 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:06.915766 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf794fd2-c9f8-4eed-afbb-1772bb55fe36 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:06.922989 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e9212-14eb-462e-8b09-2a4e334ea6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:06.923747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e9212-14eb-462e-8b09-2a4e334ea6ff admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:06.924099 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e9212-14eb-462e-8b09-2a4e334ea6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:06.924430 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e9212-14eb-462e-8b09-2a4e334ea6ff admin admin] PATH_INFO: `/v3/projects/af387a8a274b43bd8ac30e77b39cea00/users/af5bfdbdedfc496f93295ac510767625/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:06.944023 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4e9212-14eb-462e-8b09-2a4e334ea6ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=af387a8a274b43bd8ac30e77b39cea00, user_id=af5bfdbdedfc496f93295ac510767625, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:06.946927 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4e9212-14eb-462e-8b09-2a4e334ea6ff admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:06.962738 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 956/1901] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:48:06 2022] PUT /identity/v3/projects/af387a8a274b43bd8ac30e77b39cea00/users/af5bfdbdedfc496f93295ac510767625/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:48:06.974115 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020dc3e1-ca15-429d-8797-2b99b8ed3338 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:06.974411 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020dc3e1-ca15-429d-8797-2b99b8ed3338 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:06.974720 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020dc3e1-ca15-429d-8797-2b99b8ed3338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:07.015371 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-020dc3e1-ca15-429d-8797-2b99b8ed3338 None None] MFA Rules not processed for user `af5bfdbdedfc496f93295ac510767625`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:07.016609 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-020dc3e1-ca15-429d-8797-2b99b8ed3338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:07.044254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:07.092646 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 946/1902] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:48:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:48:07.097123 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e9212-14eb-462e-8b09-2a4e334ea6ff admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:07.161176 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e9212-14eb-462e-8b09-2a4e334ea6ff admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:07.166260 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946a6a7-b737-4d7b-8ed5-4159a93fd905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:07.166866 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946a6a7-b737-4d7b-8ed5-4159a93fd905 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:07.167154 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946a6a7-b737-4d7b-8ed5-4159a93fd905 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:07.167434 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946a6a7-b737-4d7b-8ed5-4159a93fd905 admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:07.168049 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6946a6a7-b737-4d7b-8ed5-4159a93fd905 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:07.170106 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6946a6a7-b737-4d7b-8ed5-4159a93fd905 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:07.196740 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 957/1903] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:48:07 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:48:07.200918 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020dc3e1-ca15-429d-8797-2b99b8ed3338 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:07.212005 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020dc3e1-ca15-429d-8797-2b99b8ed3338 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:07.267279 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf44c3a-2cfa-41e4-8b77-a01b773aba61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:07.268027 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf44c3a-2cfa-41e4-8b77-a01b773aba61 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:07.268154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf44c3a-2cfa-41e4-8b77-a01b773aba61 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:07.268422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf44c3a-2cfa-41e4-8b77-a01b773aba61 admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:07.269026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf44c3a-2cfa-41e4-8b77-a01b773aba61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:07.270990 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf44c3a-2cfa-41e4-8b77-a01b773aba61 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:07.292238 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 947/1904] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:48:07 2022] POST /identity/v3/users => generated 295 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:48:07.296338 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946a6a7-b737-4d7b-8ed5-4159a93fd905 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:07.307591 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946a6a7-b737-4d7b-8ed5-4159a93fd905 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:07.313083 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5211806-182f-4821-ab48-ffc91c8dd975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:07.313742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5211806-182f-4821-ab48-ffc91c8dd975 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:07.314025 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5211806-182f-4821-ab48-ffc91c8dd975 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:07.314300 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5211806-182f-4821-ab48-ffc91c8dd975 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:07.314940 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5211806-182f-4821-ab48-ffc91c8dd975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:07.316583 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5211806-182f-4821-ab48-ffc91c8dd975 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:07.328382 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 958/1905] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:48:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 22:48:07.332636 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf44c3a-2cfa-41e4-8b77-a01b773aba61 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:07.343204 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf44c3a-2cfa-41e4-8b77-a01b773aba61 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:07.347899 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9916c36-bce0-4903-b7d3-4d789e2c8b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:07.348555 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9916c36-bce0-4903-b7d3-4d789e2c8b50 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:07.348837 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9916c36-bce0-4903-b7d3-4d789e2c8b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:07.349143 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9916c36-bce0-4903-b7d3-4d789e2c8b50 admin admin] PATH_INFO: `/v3/projects/36313c37d427415f8ba22a16f9b00994/users/ab35efb8e8624861afc955d8280723dd/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:07.363684 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9916c36-bce0-4903-b7d3-4d789e2c8b50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36313c37d427415f8ba22a16f9b00994, user_id=ab35efb8e8624861afc955d8280723dd, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:07.367073 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9916c36-bce0-4903-b7d3-4d789e2c8b50 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:07.380870 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 948/1906] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:48:07 2022] PUT /identity/v3/projects/36313c37d427415f8ba22a16f9b00994/users/ab35efb8e8624861afc955d8280723dd/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:48:07.385249 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5211806-182f-4821-ab48-ffc91c8dd975 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:07.395613 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5211806-182f-4821-ab48-ffc91c8dd975 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:07.454612 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf6d937-6f96-42c0-a2fa-497e95d8feab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:07.455234 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6d937-6f96-42c0-a2fa-497e95d8feab admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:07.455512 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6d937-6f96-42c0-a2fa-497e95d8feab admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:07.455782 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6d937-6f96-42c0-a2fa-497e95d8feab admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:07.456418 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf6d937-6f96-42c0-a2fa-497e95d8feab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:07.458048 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf6d937-6f96-42c0-a2fa-497e95d8feab admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:07.468009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 959/1907] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:48:07 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:48:07.477129 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9916c36-bce0-4903-b7d3-4d789e2c8b50 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:07.487587 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9916c36-bce0-4903-b7d3-4d789e2c8b50 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:07.492901 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc24c94-4267-4e35-b39b-351da2580dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:07.493593 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc24c94-4267-4e35-b39b-351da2580dc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:07.493871 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc24c94-4267-4e35-b39b-351da2580dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:07.494144 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc24c94-4267-4e35-b39b-351da2580dc6 admin admin] PATH_INFO: `/v3/projects/36313c37d427415f8ba22a16f9b00994/users/ab35efb8e8624861afc955d8280723dd/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:07.496147 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc24c94-4267-4e35-b39b-351da2580dc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36313c37d427415f8ba22a16f9b00994, user_id=ab35efb8e8624861afc955d8280723dd, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:07.498876 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc24c94-4267-4e35-b39b-351da2580dc6 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:07.510124 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 949/1908] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:48:07 2022] PUT /identity/v3/projects/36313c37d427415f8ba22a16f9b00994/users/ab35efb8e8624861afc955d8280723dd/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:48:07.520013 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319ca00-5f0f-452b-b513-cd66857845c8 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:07.520325 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319ca00-5f0f-452b-b513-cd66857845c8 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:07.520601 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319ca00-5f0f-452b-b513-cd66857845c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:07.549056 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-9319ca00-5f0f-452b-b513-cd66857845c8 None None] MFA Rules not processed for user `ab35efb8e8624861afc955d8280723dd`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:07.550337 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-9319ca00-5f0f-452b-b513-cd66857845c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:07.578451 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:07.628004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 960/1909] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:48:07 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:48:07.640278 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc24c94-4267-4e35-b39b-351da2580dc6 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:07.642010 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-2fc24c94-4267-4e35-b39b-351da2580dc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:07.698799 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc24c94-4267-4e35-b39b-351da2580dc6 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:07.702535 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f83759-3062-499c-aa70-f3cd6fabfc68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:07.703134 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f83759-3062-499c-aa70-f3cd6fabfc68 service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:07.703411 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f83759-3062-499c-aa70-f3cd6fabfc68 service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:07.703683 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f83759-3062-499c-aa70-f3cd6fabfc68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:07.712733 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f83759-3062-499c-aa70-f3cd6fabfc68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:07.714184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f83759-3062-499c-aa70-f3cd6fabfc68 service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:07.723667 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 950/1910] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:48:07 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:48:07.766977 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9319ca00-5f0f-452b-b513-cd66857845c8 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:07.768500 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-9319ca00-5f0f-452b-b513-cd66857845c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:07.780433 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9319ca00-5f0f-452b-b513-cd66857845c8 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:07.784069 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57775758-82ff-446c-894c-a582f8a64b30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:07.784673 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57775758-82ff-446c-894c-a582f8a64b30 service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:07.784955 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57775758-82ff-446c-894c-a582f8a64b30 service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:07.785230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57775758-82ff-446c-894c-a582f8a64b30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:07.841173 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57775758-82ff-446c-894c-a582f8a64b30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:07.842658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57775758-82ff-446c-894c-a582f8a64b30 service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:07.852106 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 961/1911] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:48:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:48:11.688206 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f83759-3062-499c-aa70-f3cd6fabfc68 service cinder] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:11.689691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-65f83759-3062-499c-aa70-f3cd6fabfc68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:11.716172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:11.785841 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f83759-3062-499c-aa70-f3cd6fabfc68 service cinder] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:11.791261 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27905a76-9f04-4e12-8b7a-0724f298b561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:11.791904 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27905a76-9f04-4e12-8b7a-0724f298b561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:11.792224 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27905a76-9f04-4e12-8b7a-0724f298b561 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:11.792434 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27905a76-9f04-4e12-8b7a-0724f298b561 admin admin] PATH_INFO: `/v3/users/4a694b25307b4b4fb0a0efcb31b733f0` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:11.793602 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27905a76-9f04-4e12-8b7a-0724f298b561 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a694b25307b4b4fb0a0efcb31b733f0)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:11.795563 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27905a76-9f04-4e12-8b7a-0724f298b561 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:11.848411 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-27905a76-9f04-4e12-8b7a-0724f298b561 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a694b25307b4b4fb0a0efcb31b733f0'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:11.856369 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-27905a76-9f04-4e12-8b7a-0724f298b561 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a694b25307b4b4fb0a0efcb31b733f0'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:11.869181 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-27905a76-9f04-4e12-8b7a-0724f298b561 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a694b25307b4b4fb0a0efcb31b733f0'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:11.869918 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-27905a76-9f04-4e12-8b7a-0724f298b561 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a694b25307b4b4fb0a0efcb31b733f0'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:11.870571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-27905a76-9f04-4e12-8b7a-0724f298b561 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a694b25307b4b4fb0a0efcb31b733f0'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:11.884174 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 951/1912] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:48:11 2022] DELETE /identity/v3/users/4a694b25307b4b4fb0a0efcb31b733f0 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:48:11.888914 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14011398-b1e6-4dc3-8d41-0e1fc094237b None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:11.889507 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14011398-b1e6-4dc3-8d41-0e1fc094237b None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:11.889507 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14011398-b1e6-4dc3-8d41-0e1fc094237b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:11.910336 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-14011398-b1e6-4dc3-8d41-0e1fc094237b None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:11.911726 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-14011398-b1e6-4dc3-8d41-0e1fc094237b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:11.992578 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 962/1913] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:48:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:48:12.002849 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27905a76-9f04-4e12-8b7a-0724f298b561 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:12.004484 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-27905a76-9f04-4e12-8b7a-0724f298b561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:12.073201 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27905a76-9f04-4e12-8b7a-0724f298b561 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:12.076991 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c424766e-8272-4b10-adcf-fabf405ef85f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:12.077652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424766e-8272-4b10-adcf-fabf405ef85f service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:12.077932 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424766e-8272-4b10-adcf-fabf405ef85f service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:12.078202 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424766e-8272-4b10-adcf-fabf405ef85f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:12.088572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c424766e-8272-4b10-adcf-fabf405ef85f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:12.090092 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c424766e-8272-4b10-adcf-fabf405ef85f service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:12.101784 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 952/1914] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:48:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:48:12.405877 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14011398-b1e6-4dc3-8d41-0e1fc094237b None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:12.407426 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-14011398-b1e6-4dc3-8d41-0e1fc094237b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:12.422835 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14011398-b1e6-4dc3-8d41-0e1fc094237b None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:12.437662 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce587b06-1e94-48ed-b684-576599817951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:12.438319 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce587b06-1e94-48ed-b684-576599817951 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:12.438705 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce587b06-1e94-48ed-b684-576599817951 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:12.438972 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce587b06-1e94-48ed-b684-576599817951 admin admin] PATH_INFO: `/v3/projects/519c39739b7748adb8939fcfeaef57c7` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:12.440153 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce587b06-1e94-48ed-b684-576599817951 admin admin] RBAC: Authorizing `identity:delete_project(project_id=519c39739b7748adb8939fcfeaef57c7)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:12.442114 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce587b06-1e94-48ed-b684-576599817951 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:12.492771 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-ce587b06-1e94-48ed-b684-576599817951 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '519c39739b7748adb8939fcfeaef57c7', 'request_id': 'req-ce587b06-1e94-48ed-b684-576599817951'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:12.497786 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:12.499748 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-ce587b06-1e94-48ed-b684-576599817951 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '519c39739b7748adb8939fcfeaef57c7', 'request_id': 'req-ce587b06-1e94-48ed-b684-576599817951'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:12.509948 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-ce587b06-1e94-48ed-b684-576599817951 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '519c39739b7748adb8939fcfeaef57c7', 'request_id': 'req-ce587b06-1e94-48ed-b684-576599817951'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:12.523163 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-ce587b06-1e94-48ed-b684-576599817951 admin admin] The token cache is being invalidate because project 519c39739b7748adb8939fcfeaef57c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:48:12.523452 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-ce587b06-1e94-48ed-b684-576599817951 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:12.524297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-ce587b06-1e94-48ed-b684-576599817951 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:12.525850 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 963/1915] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:48:12 2022] DELETE /identity/v3/projects/519c39739b7748adb8939fcfeaef57c7 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:48:12.530676 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40df929-c2a9-471f-8a00-7bebc8446b53 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:12.530966 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40df929-c2a9-471f-8a00-7bebc8446b53 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:12.531195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40df929-c2a9-471f-8a00-7bebc8446b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:12.552200 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-e40df929-c2a9-471f-8a00-7bebc8446b53 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:12.553766 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-e40df929-c2a9-471f-8a00-7bebc8446b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:12.590681 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:12.627539 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 953/1916] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:48:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:48:12.635439 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e95fe-deb1-4870-a3bf-0cd124b4616d None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:12.635734 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e95fe-deb1-4870-a3bf-0cd124b4616d None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:12.636016 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e95fe-deb1-4870-a3bf-0cd124b4616d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:12.667383 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-413e95fe-deb1-4870-a3bf-0cd124b4616d None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:12.669229 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-413e95fe-deb1-4870-a3bf-0cd124b4616d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:12.686415 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 964/1917] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:48:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:48:12.691031 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40df929-c2a9-471f-8a00-7bebc8446b53 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:12.702634 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40df929-c2a9-471f-8a00-7bebc8446b53 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:12.707729 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0631f071-1492-4f85-988f-304349ab880e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:12.708349 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631f071-1492-4f85-988f-304349ab880e admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:12.709244 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631f071-1492-4f85-988f-304349ab880e admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:12.709472 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631f071-1492-4f85-988f-304349ab880e admin admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:12.710242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0631f071-1492-4f85-988f-304349ab880e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:48:12.710705 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0631f071-1492-4f85-988f-304349ab880e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:12.712877 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0631f071-1492-4f85-988f-304349ab880e admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:12.727623 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 954/1918] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:48:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:48:12.732994 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413e95fe-deb1-4870-a3bf-0cd124b4616d None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:12.745384 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413e95fe-deb1-4870-a3bf-0cd124b4616d None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:12.751338 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43121bd8-831b-4010-bdfc-a3da0481eaea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:12.753922 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43121bd8-831b-4010-bdfc-a3da0481eaea admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:12.754532 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43121bd8-831b-4010-bdfc-a3da0481eaea admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:12.754945 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43121bd8-831b-4010-bdfc-a3da0481eaea admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:12.756022 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43121bd8-831b-4010-bdfc-a3da0481eaea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:12.758736 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43121bd8-831b-4010-bdfc-a3da0481eaea admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:12.792906 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 965/1919] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:48:12 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:48:12.797130 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0631f071-1492-4f85-988f-304349ab880e admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:12.808157 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0631f071-1492-4f85-988f-304349ab880e admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:12.883190 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bdf94-0a60-44a4-bb87-6fea30811a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:12.884452 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bdf94-0a60-44a4-bb87-6fea30811a20 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:12.885010 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bdf94-0a60-44a4-bb87-6fea30811a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:12.885605 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bdf94-0a60-44a4-bb87-6fea30811a20 admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:12.886986 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19bdf94-0a60-44a4-bb87-6fea30811a20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:12.891162 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19bdf94-0a60-44a4-bb87-6fea30811a20 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:12.926082 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 955/1920] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:48:12 2022] POST /identity/v3/users => generated 288 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:48:12.931808 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43121bd8-831b-4010-bdfc-a3da0481eaea admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:12.945953 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43121bd8-831b-4010-bdfc-a3da0481eaea admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:12.951693 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118cfe0b-f354-4415-af9b-242799eb496a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:12.952878 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118cfe0b-f354-4415-af9b-242799eb496a admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:12.953519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118cfe0b-f354-4415-af9b-242799eb496a admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:12.953940 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118cfe0b-f354-4415-af9b-242799eb496a admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:12.954890 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118cfe0b-f354-4415-af9b-242799eb496a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:12.957099 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118cfe0b-f354-4415-af9b-242799eb496a admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:12.973320 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 966/1921] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:48:12 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:48:12.976185 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bdf94-0a60-44a4-bb87-6fea30811a20 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:12.988876 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bdf94-0a60-44a4-bb87-6fea30811a20 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:12.994212 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d1414-2d83-46ad-b652-6603c643004d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:12.994891 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d1414-2d83-46ad-b652-6603c643004d admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:12.995241 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d1414-2d83-46ad-b652-6603c643004d admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:12.995585 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d1414-2d83-46ad-b652-6603c643004d admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:12.996291 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d1414-2d83-46ad-b652-6603c643004d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:12.998175 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d1414-2d83-46ad-b652-6603c643004d admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:13.011795 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 956/1922] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:48:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 22:48:13.019080 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118cfe0b-f354-4415-af9b-242799eb496a admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:13.030741 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118cfe0b-f354-4415-af9b-242799eb496a admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:13.036431 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abea902-ee19-4f1e-98af-d81a942944cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:13.037551 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abea902-ee19-4f1e-98af-d81a942944cf admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:13.037692 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abea902-ee19-4f1e-98af-d81a942944cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:13.038052 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abea902-ee19-4f1e-98af-d81a942944cf admin admin] PATH_INFO: `/v3/projects/427b44e6dcfe476e8a895ae972db0534/users/1dbcc417dbe244cd9e5c780055b839bb/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:13.054251 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abea902-ee19-4f1e-98af-d81a942944cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=427b44e6dcfe476e8a895ae972db0534, user_id=1dbcc417dbe244cd9e5c780055b839bb, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:13.057635 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abea902-ee19-4f1e-98af-d81a942944cf admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:13.072550 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 967/1923] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:48:13 2022] PUT /identity/v3/projects/427b44e6dcfe476e8a895ae972db0534/users/1dbcc417dbe244cd9e5c780055b839bb/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:48:13.084812 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e726d94-9382-453e-a164-31808dd08224 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:13.085151 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e726d94-9382-453e-a164-31808dd08224 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:13.085468 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e726d94-9382-453e-a164-31808dd08224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:13.115132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-9e726d94-9382-453e-a164-31808dd08224 None None] MFA Rules not processed for user `1dbcc417dbe244cd9e5c780055b839bb`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:13.116397 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-9e726d94-9382-453e-a164-31808dd08224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:13.201150 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 957/1924] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:48:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:48:13.205728 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abea902-ee19-4f1e-98af-d81a942944cf admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:13.306711 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abea902-ee19-4f1e-98af-d81a942944cf admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:13.312550 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d43ff-bbe2-4373-93bf-5318fcac7500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:13.313320 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d43ff-bbe2-4373-93bf-5318fcac7500 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:13.313620 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d43ff-bbe2-4373-93bf-5318fcac7500 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:13.313900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d43ff-bbe2-4373-93bf-5318fcac7500 admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:13.314541 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d43ff-bbe2-4373-93bf-5318fcac7500 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:13.316483 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d43ff-bbe2-4373-93bf-5318fcac7500 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:13.354951 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 968/1925] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:48:13 2022] POST /identity/v3/projects => generated 375 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:48:13.359265 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e726d94-9382-453e-a164-31808dd08224 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:13.370663 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e726d94-9382-453e-a164-31808dd08224 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:13.431354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:13.465093 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f1748-6458-4423-bc1e-fd0f0708ddc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:13.465768 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f1748-6458-4423-bc1e-fd0f0708ddc3 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:13.466161 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f1748-6458-4423-bc1e-fd0f0708ddc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:13.466451 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f1748-6458-4423-bc1e-fd0f0708ddc3 admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:13.467081 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761f1748-6458-4423-bc1e-fd0f0708ddc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:13.469022 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761f1748-6458-4423-bc1e-fd0f0708ddc3 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:13.495443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 958/1926] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:48:13 2022] POST /identity/v3/users => generated 294 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:48:13.499744 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d43ff-bbe2-4373-93bf-5318fcac7500 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:13.513192 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d43ff-bbe2-4373-93bf-5318fcac7500 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:13.519593 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f2121-8bc8-48d7-94ea-728a51ae0a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:13.522526 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f2121-8bc8-48d7-94ea-728a51ae0a1f admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:13.522890 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f2121-8bc8-48d7-94ea-728a51ae0a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:13.523245 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f2121-8bc8-48d7-94ea-728a51ae0a1f admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:13.523920 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f2121-8bc8-48d7-94ea-728a51ae0a1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:13.525698 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f2121-8bc8-48d7-94ea-728a51ae0a1f admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:13.538288 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 969/1927] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 22:48:13.542813 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f1748-6458-4423-bc1e-fd0f0708ddc3 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:13.554055 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f1748-6458-4423-bc1e-fd0f0708ddc3 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:13.562374 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a1871b-69cc-4695-9470-5e3536b4804b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:13.563102 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1871b-69cc-4695-9470-5e3536b4804b admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:13.563382 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1871b-69cc-4695-9470-5e3536b4804b admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:13.563679 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1871b-69cc-4695-9470-5e3536b4804b admin admin] PATH_INFO: `/v3/projects/35341b5eaded4998a70c7d5a844358b5/users/4dc8239481df43e1b95f8d5613a4b253/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:13.581470 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a1871b-69cc-4695-9470-5e3536b4804b admin admin] RBAC: Authorizing `identity:create_grant(project_id=35341b5eaded4998a70c7d5a844358b5, user_id=4dc8239481df43e1b95f8d5613a4b253, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:13.584733 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a1871b-69cc-4695-9470-5e3536b4804b admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:13.601548 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 959/1928] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:48:13 2022] PUT /identity/v3/projects/35341b5eaded4998a70c7d5a844358b5/users/4dc8239481df43e1b95f8d5613a4b253/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:48:13.605822 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f2121-8bc8-48d7-94ea-728a51ae0a1f admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:13.617716 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f2121-8bc8-48d7-94ea-728a51ae0a1f admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:13.682380 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:13.691522 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e624fb0-6454-4bac-ba31-78e323217712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:13.692259 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e624fb0-6454-4bac-ba31-78e323217712 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:13.692723 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e624fb0-6454-4bac-ba31-78e323217712 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:13.693608 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e624fb0-6454-4bac-ba31-78e323217712 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:13.695231 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e624fb0-6454-4bac-ba31-78e323217712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:13.697316 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e624fb0-6454-4bac-ba31-78e323217712 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:13.725842 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 970/1929] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:48:13.727464 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a1871b-69cc-4695-9470-5e3536b4804b admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:13.743432 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a1871b-69cc-4695-9470-5e3536b4804b admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:13.750510 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333095bb-5e25-46ab-93c8-a99981ee5993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:13.751103 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333095bb-5e25-46ab-93c8-a99981ee5993 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:13.751463 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333095bb-5e25-46ab-93c8-a99981ee5993 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:13.751819 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333095bb-5e25-46ab-93c8-a99981ee5993 admin admin] PATH_INFO: `/v3/projects/35341b5eaded4998a70c7d5a844358b5/users/4dc8239481df43e1b95f8d5613a4b253/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:13.753995 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333095bb-5e25-46ab-93c8-a99981ee5993 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35341b5eaded4998a70c7d5a844358b5, user_id=4dc8239481df43e1b95f8d5613a4b253, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:13.758539 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333095bb-5e25-46ab-93c8-a99981ee5993 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:13.772137 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 960/1930] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:48:13 2022] PUT /identity/v3/projects/35341b5eaded4998a70c7d5a844358b5/users/4dc8239481df43e1b95f8d5613a4b253/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:48:13.781976 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341e9de-6b7a-478b-a53f-ab9f561a49e2 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:13.782346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341e9de-6b7a-478b-a53f-ab9f561a49e2 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:13.782709 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341e9de-6b7a-478b-a53f-ab9f561a49e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:13.823080 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-e341e9de-6b7a-478b-a53f-ab9f561a49e2 None None] MFA Rules not processed for user `4dc8239481df43e1b95f8d5613a4b253`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:13.824558 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-e341e9de-6b7a-478b-a53f-ab9f561a49e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:13.915341 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 971/1931] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:48:13 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:48:13.926398 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333095bb-5e25-46ab-93c8-a99981ee5993 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:13.928709 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-333095bb-5e25-46ab-93c8-a99981ee5993 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:14.058480 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333095bb-5e25-46ab-93c8-a99981ee5993 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:14.070959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526da194-d742-459e-9ff3-70cfc20a54f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:14.071795 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526da194-d742-459e-9ff3-70cfc20a54f6 service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:14.072392 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526da194-d742-459e-9ff3-70cfc20a54f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:14.072678 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526da194-d742-459e-9ff3-70cfc20a54f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:14.177017 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341e9de-6b7a-478b-a53f-ab9f561a49e2 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:14.179477 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-e341e9de-6b7a-478b-a53f-ab9f561a49e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:14.182561 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526da194-d742-459e-9ff3-70cfc20a54f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:14.186313 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526da194-d742-459e-9ff3-70cfc20a54f6 service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:14.200735 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 961/1932] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:48:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:48:14.288462 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341e9de-6b7a-478b-a53f-ab9f561a49e2 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:14.296909 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde58be-5b2a-498b-b53a-def595401d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:14.297612 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde58be-5b2a-498b-b53a-def595401d70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:14.297941 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde58be-5b2a-498b-b53a-def595401d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:14.298214 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde58be-5b2a-498b-b53a-def595401d70 admin admin] PATH_INFO: `/v3/users/6cf05194be26416eb5b9f03103603b04` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:14.299586 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde58be-5b2a-498b-b53a-def595401d70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cf05194be26416eb5b9f03103603b04)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:14.301662 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde58be-5b2a-498b-b53a-def595401d70 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:14.403800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-2bde58be-5b2a-498b-b53a-def595401d70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cf05194be26416eb5b9f03103603b04'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:14.422191 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-2bde58be-5b2a-498b-b53a-def595401d70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cf05194be26416eb5b9f03103603b04'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:14.437128 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-2bde58be-5b2a-498b-b53a-def595401d70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cf05194be26416eb5b9f03103603b04'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:14.438830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-2bde58be-5b2a-498b-b53a-def595401d70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cf05194be26416eb5b9f03103603b04'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:14.442929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-2bde58be-5b2a-498b-b53a-def595401d70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cf05194be26416eb5b9f03103603b04'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:14.465333 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:14.483259 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 972/1933] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:48:14 2022] DELETE /identity/v3/users/6cf05194be26416eb5b9f03103603b04 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:48:14.487960 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ff785-7e68-4e0b-adc8-7d5a066143f2 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:14.488252 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ff785-7e68-4e0b-adc8-7d5a066143f2 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:14.488534 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ff785-7e68-4e0b-adc8-7d5a066143f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:14.535579 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-730ff785-7e68-4e0b-adc8-7d5a066143f2 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:14.541410 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-730ff785-7e68-4e0b-adc8-7d5a066143f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:14.658631 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:14.712953 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 962/1934] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:48:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:48:14.742177 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde58be-5b2a-498b-b53a-def595401d70 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:14.744119 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-2bde58be-5b2a-498b-b53a-def595401d70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:14.931670 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde58be-5b2a-498b-b53a-def595401d70 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:14.940709 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc7e294-6a6f-4a46-b960-0ccb79c08728 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:14.942090 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7e294-6a6f-4a46-b960-0ccb79c08728 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:14.942484 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7e294-6a6f-4a46-b960-0ccb79c08728 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:14.942895 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7e294-6a6f-4a46-b960-0ccb79c08728 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:14.957467 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc7e294-6a6f-4a46-b960-0ccb79c08728 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:14.960328 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc7e294-6a6f-4a46-b960-0ccb79c08728 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:14.974001 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 973/1935] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:48:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:48:15.025704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ff785-7e68-4e0b-adc8-7d5a066143f2 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:15.028307 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-730ff785-7e68-4e0b-adc8-7d5a066143f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:15.054651 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ff785-7e68-4e0b-adc8-7d5a066143f2 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:15.065528 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6eb04e-bffd-4f1d-8812-9ddf27b1547c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:15.066542 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6eb04e-bffd-4f1d-8812-9ddf27b1547c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:15.067061 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6eb04e-bffd-4f1d-8812-9ddf27b1547c admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:15.067617 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6eb04e-bffd-4f1d-8812-9ddf27b1547c admin admin] PATH_INFO: `/v3/users/fa893fb96e354b76b3a6b59f3e4249f7` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:15.069407 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6eb04e-bffd-4f1d-8812-9ddf27b1547c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa893fb96e354b76b3a6b59f3e4249f7)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:15.071723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6eb04e-bffd-4f1d-8812-9ddf27b1547c admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:15.147469 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-0a6eb04e-bffd-4f1d-8812-9ddf27b1547c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa893fb96e354b76b3a6b59f3e4249f7'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:15.156785 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-0a6eb04e-bffd-4f1d-8812-9ddf27b1547c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa893fb96e354b76b3a6b59f3e4249f7'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:15.172787 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-0a6eb04e-bffd-4f1d-8812-9ddf27b1547c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa893fb96e354b76b3a6b59f3e4249f7'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:15.174219 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-0a6eb04e-bffd-4f1d-8812-9ddf27b1547c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa893fb96e354b76b3a6b59f3e4249f7'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:15.175662 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-0a6eb04e-bffd-4f1d-8812-9ddf27b1547c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa893fb96e354b76b3a6b59f3e4249f7'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:15.205636 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd74915-245f-4776-859a-e1e09458e27b None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:15.206183 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd74915-245f-4776-859a-e1e09458e27b None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:15.206434 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd74915-245f-4776-859a-e1e09458e27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:15.210183 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 963/1936] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:48:15 2022] DELETE /identity/v3/users/fa893fb96e354b76b3a6b59f3e4249f7 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:48:15.231052 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-edd74915-245f-4776-859a-e1e09458e27b None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:15.233028 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-edd74915-245f-4776-859a-e1e09458e27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:15.317793 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 974/1937] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:48:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:48:15.357609 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6eb04e-bffd-4f1d-8812-9ddf27b1547c admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:15.367250 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-0a6eb04e-bffd-4f1d-8812-9ddf27b1547c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:15.443812 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6eb04e-bffd-4f1d-8812-9ddf27b1547c admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:15.449155 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e692be55-a2c5-452f-8092-1c3338f68b51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:15.450078 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e692be55-a2c5-452f-8092-1c3338f68b51 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:15.450450 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e692be55-a2c5-452f-8092-1c3338f68b51 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:15.450814 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e692be55-a2c5-452f-8092-1c3338f68b51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:15.462460 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e692be55-a2c5-452f-8092-1c3338f68b51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:15.464114 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e692be55-a2c5-452f-8092-1c3338f68b51 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:15.476712 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 964/1938] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:48:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:48:15.810950 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd74915-245f-4776-859a-e1e09458e27b None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:15.817206 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-edd74915-245f-4776-859a-e1e09458e27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:15.852583 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd74915-245f-4776-859a-e1e09458e27b None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:15.866487 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe623c-02db-4f11-b2ed-9d080bf11707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:15.867206 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe623c-02db-4f11-b2ed-9d080bf11707 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:15.867496 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe623c-02db-4f11-b2ed-9d080bf11707 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:15.867908 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe623c-02db-4f11-b2ed-9d080bf11707 admin admin] PATH_INFO: `/v3/projects/ca7c14f24ef743649ef07406dc897237` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:15.869920 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe623c-02db-4f11-b2ed-9d080bf11707 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca7c14f24ef743649ef07406dc897237)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:15.872217 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe623c-02db-4f11-b2ed-9d080bf11707 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:15.984710 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-e1fe623c-02db-4f11-b2ed-9d080bf11707 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca7c14f24ef743649ef07406dc897237', 'request_id': 'req-e1fe623c-02db-4f11-b2ed-9d080bf11707'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:15.996562 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-e1fe623c-02db-4f11-b2ed-9d080bf11707 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca7c14f24ef743649ef07406dc897237', 'request_id': 'req-e1fe623c-02db-4f11-b2ed-9d080bf11707'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:16.009654 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-e1fe623c-02db-4f11-b2ed-9d080bf11707 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca7c14f24ef743649ef07406dc897237', 'request_id': 'req-e1fe623c-02db-4f11-b2ed-9d080bf11707'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:16.048674 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-e1fe623c-02db-4f11-b2ed-9d080bf11707 admin admin] The token cache is being invalidate because project ca7c14f24ef743649ef07406dc897237 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:48:16.048979 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-e1fe623c-02db-4f11-b2ed-9d080bf11707 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:16.049684 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-e1fe623c-02db-4f11-b2ed-9d080bf11707 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:16.054512 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 975/1939] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:48:15 2022] DELETE /identity/v3/projects/ca7c14f24ef743649ef07406dc897237 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:48:16.060868 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e692be55-a2c5-452f-8092-1c3338f68b51 service neutron] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:16.076692 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e692be55-a2c5-452f-8092-1c3338f68b51 service neutron] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:16.192207 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:16.200945 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e2bae-b562-4395-b417-5f7a687d318f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:16.203476 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e2bae-b562-4395-b417-5f7a687d318f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:16.203762 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e2bae-b562-4395-b417-5f7a687d318f admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:16.204090 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e2bae-b562-4395-b417-5f7a687d318f admin admin] PATH_INFO: `/v3/users/af5bfdbdedfc496f93295ac510767625` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:16.205735 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338e2bae-b562-4395-b417-5f7a687d318f admin admin] RBAC: Authorizing `identity:delete_user(user_id=af5bfdbdedfc496f93295ac510767625)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:16.208773 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338e2bae-b562-4395-b417-5f7a687d318f admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:16.263631 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe623c-02db-4f11-b2ed-9d080bf11707 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:16.266679 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-e1fe623c-02db-4f11-b2ed-9d080bf11707 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:16.300742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe623c-02db-4f11-b2ed-9d080bf11707 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:16.311903 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f71d59-edae-4190-9316-1f6bc2e9974c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:16.313942 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f71d59-edae-4190-9316-1f6bc2e9974c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:16.315516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f71d59-edae-4190-9316-1f6bc2e9974c admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:16.316642 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f71d59-edae-4190-9316-1f6bc2e9974c admin admin] PATH_INFO: `/v3/projects/a55e5a8aae6e497db0fc4be4347021e0` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:16.320774 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-338e2bae-b562-4395-b417-5f7a687d318f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af5bfdbdedfc496f93295ac510767625'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:16.321153 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f71d59-edae-4190-9316-1f6bc2e9974c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a55e5a8aae6e497db0fc4be4347021e0)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:16.327680 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f71d59-edae-4190-9316-1f6bc2e9974c admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:16.342903 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-338e2bae-b562-4395-b417-5f7a687d318f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af5bfdbdedfc496f93295ac510767625'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:16.393063 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-338e2bae-b562-4395-b417-5f7a687d318f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af5bfdbdedfc496f93295ac510767625'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:16.397485 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-338e2bae-b562-4395-b417-5f7a687d318f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af5bfdbdedfc496f93295ac510767625'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:16.399469 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-338e2bae-b562-4395-b417-5f7a687d318f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af5bfdbdedfc496f93295ac510767625'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:16.408253 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:16.439852 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 965/1940] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:48:16 2022] DELETE /identity/v3/users/af5bfdbdedfc496f93295ac510767625 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:48:16.491901 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-73f71d59-edae-4190-9316-1f6bc2e9974c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a55e5a8aae6e497db0fc4be4347021e0', 'request_id': 'req-73f71d59-edae-4190-9316-1f6bc2e9974c'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:16.501629 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-73f71d59-edae-4190-9316-1f6bc2e9974c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a55e5a8aae6e497db0fc4be4347021e0', 'request_id': 'req-73f71d59-edae-4190-9316-1f6bc2e9974c'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:16.528777 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-73f71d59-edae-4190-9316-1f6bc2e9974c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a55e5a8aae6e497db0fc4be4347021e0', 'request_id': 'req-73f71d59-edae-4190-9316-1f6bc2e9974c'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:16.550426 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-73f71d59-edae-4190-9316-1f6bc2e9974c admin admin] The token cache is being invalidate because project a55e5a8aae6e497db0fc4be4347021e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:48:16.550962 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-73f71d59-edae-4190-9316-1f6bc2e9974c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:16.551851 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-73f71d59-edae-4190-9316-1f6bc2e9974c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:16.555156 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 976/1941] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:48:16 2022] DELETE /identity/v3/projects/a55e5a8aae6e497db0fc4be4347021e0 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:48:16.559730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e2bae-b562-4395-b417-5f7a687d318f admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:16.561856 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-338e2bae-b562-4395-b417-5f7a687d318f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:16.665046 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e2bae-b562-4395-b417-5f7a687d318f admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:16.862237 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a710d4e-5f1f-4968-bfdc-6c694b7a88b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:16.862907 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a710d4e-5f1f-4968-bfdc-6c694b7a88b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:16.863188 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a710d4e-5f1f-4968-bfdc-6c694b7a88b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:16.863460 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a710d4e-5f1f-4968-bfdc-6c694b7a88b7 admin admin] PATH_INFO: `/v3/users/47491921214d46559d97c8bd5335c45d` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:16.870110 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a710d4e-5f1f-4968-bfdc-6c694b7a88b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47491921214d46559d97c8bd5335c45d)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:16.874637 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a710d4e-5f1f-4968-bfdc-6c694b7a88b7 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:16.908870 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:16.934244 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-8a710d4e-5f1f-4968-bfdc-6c694b7a88b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47491921214d46559d97c8bd5335c45d'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:16.941328 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-8a710d4e-5f1f-4968-bfdc-6c694b7a88b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47491921214d46559d97c8bd5335c45d'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:16.958386 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-8a710d4e-5f1f-4968-bfdc-6c694b7a88b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47491921214d46559d97c8bd5335c45d'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:16.959193 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-8a710d4e-5f1f-4968-bfdc-6c694b7a88b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47491921214d46559d97c8bd5335c45d'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:16.960855 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-8a710d4e-5f1f-4968-bfdc-6c694b7a88b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47491921214d46559d97c8bd5335c45d'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:16.980171 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 966/1942] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:48:16 2022] DELETE /identity/v3/users/47491921214d46559d97c8bd5335c45d => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:48:17.199649 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f71d59-edae-4190-9316-1f6bc2e9974c admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:17.201129 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-73f71d59-edae-4190-9316-1f6bc2e9974c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:17.204177 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003157c-335c-46d3-a5db-928f10128ae4 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:17.204666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003157c-335c-46d3-a5db-928f10128ae4 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:17.204744 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003157c-335c-46d3-a5db-928f10128ae4 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:17.207446 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 967/1943] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 10 22:48:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:48:17.215653 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f71d59-edae-4190-9316-1f6bc2e9974c admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:17.215857 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef502c4-5749-48ea-8417-2358e7173b00 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:17.216196 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef502c4-5749-48ea-8417-2358e7173b00 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:17.216750 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef502c4-5749-48ea-8417-2358e7173b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:17.245436 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-eef502c4-5749-48ea-8417-2358e7173b00 None None] MFA Rules not processed for user `b22838a8ad3949e59cdbaaa1801cd07f`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:17.246834 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-eef502c4-5749-48ea-8417-2358e7173b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:17.303341 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e616f-ede7-4af7-a11a-ad9c3a9d466e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:17.304282 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e616f-ede7-4af7-a11a-ad9c3a9d466e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:17.304282 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e616f-ede7-4af7-a11a-ad9c3a9d466e admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:17.304539 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e616f-ede7-4af7-a11a-ad9c3a9d466e admin admin] PATH_INFO: `/v3/projects/af387a8a274b43bd8ac30e77b39cea00` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:17.305737 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975e616f-ede7-4af7-a11a-ad9c3a9d466e admin admin] RBAC: Authorizing `identity:delete_project(project_id=af387a8a274b43bd8ac30e77b39cea00)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:17.307872 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975e616f-ede7-4af7-a11a-ad9c3a9d466e admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:17.345486 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 968/1944] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 10 22:48:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:48:17.354440 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef502c4-5749-48ea-8417-2358e7173b00 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:17.359059 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-eef502c4-5749-48ea-8417-2358e7173b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:17.367805 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-975e616f-ede7-4af7-a11a-ad9c3a9d466e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af387a8a274b43bd8ac30e77b39cea00', 'request_id': 'req-975e616f-ede7-4af7-a11a-ad9c3a9d466e'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:17.373974 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef502c4-5749-48ea-8417-2358e7173b00 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:17.375390 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-975e616f-ede7-4af7-a11a-ad9c3a9d466e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af387a8a274b43bd8ac30e77b39cea00', 'request_id': 'req-975e616f-ede7-4af7-a11a-ad9c3a9d466e'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:17.385507 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-975e616f-ede7-4af7-a11a-ad9c3a9d466e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af387a8a274b43bd8ac30e77b39cea00', 'request_id': 'req-975e616f-ede7-4af7-a11a-ad9c3a9d466e'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:17.399946 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-975e616f-ede7-4af7-a11a-ad9c3a9d466e admin admin] The token cache is being invalidate because project af387a8a274b43bd8ac30e77b39cea00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:48:17.400235 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-975e616f-ede7-4af7-a11a-ad9c3a9d466e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:17.400927 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-975e616f-ede7-4af7-a11a-ad9c3a9d466e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:17.402470 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 977/1945] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:48:17 2022] DELETE /identity/v3/projects/af387a8a274b43bd8ac30e77b39cea00 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:48:17.403715 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e616f-ede7-4af7-a11a-ad9c3a9d466e admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:17.405525 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-975e616f-ede7-4af7-a11a-ad9c3a9d466e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:17.447692 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:17.448604 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311bd4d6-00e2-4e39-b41f-8e4198a644c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:17.449341 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bd4d6-00e2-4e39-b41f-8e4198a644c6 service nova] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:17.449663 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bd4d6-00e2-4e39-b41f-8e4198a644c6 service nova] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:17.449961 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bd4d6-00e2-4e39-b41f-8e4198a644c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:17.451758 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-311bd4d6-00e2-4e39-b41f-8e4198a644c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:17.464259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311bd4d6-00e2-4e39-b41f-8e4198a644c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:17.465905 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311bd4d6-00e2-4e39-b41f-8e4198a644c6 service nova] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:17.476901 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e616f-ede7-4af7-a11a-ad9c3a9d466e admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:17.477048 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 969/1946] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:48:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:48:17.477729 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311bd4d6-00e2-4e39-b41f-8e4198a644c6 service nova] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:17.479127 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-311bd4d6-00e2-4e39-b41f-8e4198a644c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:17.486400 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0488ad10-41f6-499e-bb48-5b15495db31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:17.487112 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488ad10-41f6-499e-bb48-5b15495db31b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:17.487397 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488ad10-41f6-499e-bb48-5b15495db31b admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:17.487672 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488ad10-41f6-499e-bb48-5b15495db31b admin admin] PATH_INFO: `/v3/projects/a22a97cbd7ae4efe9cd3b7d7206ead52` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:17.488835 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0488ad10-41f6-499e-bb48-5b15495db31b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a22a97cbd7ae4efe9cd3b7d7206ead52)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:17.491528 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0488ad10-41f6-499e-bb48-5b15495db31b admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:17.493605 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311bd4d6-00e2-4e39-b41f-8e4198a644c6 service nova] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:17.498601 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59adeed-c3c9-45ee-b038-64bd47d0b586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:17.498986 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59adeed-c3c9-45ee-b038-64bd47d0b586 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:17.499332 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59adeed-c3c9-45ee-b038-64bd47d0b586 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:17.499673 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59adeed-c3c9-45ee-b038-64bd47d0b586 admin admin] PATH_INFO: `/v3/users/ab35efb8e8624861afc955d8280723dd` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:17.500869 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59adeed-c3c9-45ee-b038-64bd47d0b586 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab35efb8e8624861afc955d8280723dd)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:17.502808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59adeed-c3c9-45ee-b038-64bd47d0b586 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:17.519295 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:17.550771 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-0488ad10-41f6-499e-bb48-5b15495db31b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a22a97cbd7ae4efe9cd3b7d7206ead52', 'request_id': 'req-0488ad10-41f6-499e-bb48-5b15495db31b'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:17.563366 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-0488ad10-41f6-499e-bb48-5b15495db31b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a22a97cbd7ae4efe9cd3b7d7206ead52', 'request_id': 'req-0488ad10-41f6-499e-bb48-5b15495db31b'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:17.564413 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e59adeed-c3c9-45ee-b038-64bd47d0b586 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab35efb8e8624861afc955d8280723dd'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:17.573172 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-0488ad10-41f6-499e-bb48-5b15495db31b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a22a97cbd7ae4efe9cd3b7d7206ead52', 'request_id': 'req-0488ad10-41f6-499e-bb48-5b15495db31b'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:17.573848 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e59adeed-c3c9-45ee-b038-64bd47d0b586 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab35efb8e8624861afc955d8280723dd'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:17.586157 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-0488ad10-41f6-499e-bb48-5b15495db31b admin admin] The token cache is being invalidate because project a22a97cbd7ae4efe9cd3b7d7206ead52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:48:17.586573 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-0488ad10-41f6-499e-bb48-5b15495db31b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:17.587288 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-0488ad10-41f6-499e-bb48-5b15495db31b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:17.589131 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 978/1947] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:48:17 2022] DELETE /identity/v3/projects/a22a97cbd7ae4efe9cd3b7d7206ead52 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:48:17.589864 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0488ad10-41f6-499e-bb48-5b15495db31b admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:17.591207 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-0488ad10-41f6-499e-bb48-5b15495db31b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:17.591769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e59adeed-c3c9-45ee-b038-64bd47d0b586 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab35efb8e8624861afc955d8280723dd'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:17.592725 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e59adeed-c3c9-45ee-b038-64bd47d0b586 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab35efb8e8624861afc955d8280723dd'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:17.593493 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e59adeed-c3c9-45ee-b038-64bd47d0b586 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab35efb8e8624861afc955d8280723dd'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:17.608757 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 970/1948] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:48:17 2022] DELETE /identity/v3/users/ab35efb8e8624861afc955d8280723dd => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:48:17.610068 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a00aa-ca70-40ce-b8a4-424cb3deb8ed None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:17.610354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a00aa-ca70-40ce-b8a4-424cb3deb8ed None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:17.610630 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a00aa-ca70-40ce-b8a4-424cb3deb8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:17.636017 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-4d9a00aa-ca70-40ce-b8a4-424cb3deb8ed None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:17.637293 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-4d9a00aa-ca70-40ce-b8a4-424cb3deb8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:17.681363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0488ad10-41f6-499e-bb48-5b15495db31b admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:17.750744 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 971/1949] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:48:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:48:17.759234 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1546a-de21-49d8-adc1-45bf87545d57 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:17.759676 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1546a-de21-49d8-adc1-45bf87545d57 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:17.760236 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1546a-de21-49d8-adc1-45bf87545d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:17.774032 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2fa4a-6f63-4f7f-bda4-f55c45430161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:17.774769 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2fa4a-6f63-4f7f-bda4-f55c45430161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:17.775114 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2fa4a-6f63-4f7f-bda4-f55c45430161 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:17.775459 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2fa4a-6f63-4f7f-bda4-f55c45430161 admin admin] PATH_INFO: `/v3/users/1dbcc417dbe244cd9e5c780055b839bb` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:17.777137 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de2fa4a-6f63-4f7f-bda4-f55c45430161 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dbcc417dbe244cd9e5c780055b839bb)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:17.779438 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de2fa4a-6f63-4f7f-bda4-f55c45430161 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:17.793451 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-99a1546a-de21-49d8-adc1-45bf87545d57 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:17.797880 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-99a1546a-de21-49d8-adc1-45bf87545d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:17.804414 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:17.820295 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 972/1950] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:48:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:48:17.825959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1546a-de21-49d8-adc1-45bf87545d57 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:17.842084 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1546a-de21-49d8-adc1-45bf87545d57 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:17.850662 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084acb2-d903-4efd-aea7-9b7118b1bb3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:17.868326 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084acb2-d903-4efd-aea7-9b7118b1bb3d admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:17.868780 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084acb2-d903-4efd-aea7-9b7118b1bb3d admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:17.869464 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084acb2-d903-4efd-aea7-9b7118b1bb3d admin admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:17.870289 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084acb2-d903-4efd-aea7-9b7118b1bb3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:48:17.871760 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084acb2-d903-4efd-aea7-9b7118b1bb3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:17.877639 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084acb2-d903-4efd-aea7-9b7118b1bb3d admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:17.883956 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-8de2fa4a-6f63-4f7f-bda4-f55c45430161 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dbcc417dbe244cd9e5c780055b839bb'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:17.900169 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-8de2fa4a-6f63-4f7f-bda4-f55c45430161 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dbcc417dbe244cd9e5c780055b839bb'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:17.910924 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 973/1951] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:48:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:48:17.915641 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084acb2-d903-4efd-aea7-9b7118b1bb3d admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:17.924476 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-8de2fa4a-6f63-4f7f-bda4-f55c45430161 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dbcc417dbe244cd9e5c780055b839bb'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:17.926699 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-8de2fa4a-6f63-4f7f-bda4-f55c45430161 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dbcc417dbe244cd9e5c780055b839bb'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:17.928605 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-8de2fa4a-6f63-4f7f-bda4-f55c45430161 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dbcc417dbe244cd9e5c780055b839bb'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:17.944387 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084acb2-d903-4efd-aea7-9b7118b1bb3d admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:17.959818 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 979/1952] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:48:17 2022] DELETE /identity/v3/users/1dbcc417dbe244cd9e5c780055b839bb => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:48:17.962485 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f0127-de39-48f8-bab9-0f9585935fb1 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:17.962854 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df9482-31d2-48cd-8bd5-a215c38ea97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:17.963484 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df9482-31d2-48cd-8bd5-a215c38ea97b admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:17.963660 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f0127-de39-48f8-bab9-0f9585935fb1 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:17.963791 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df9482-31d2-48cd-8bd5-a215c38ea97b admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:17.964113 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df9482-31d2-48cd-8bd5-a215c38ea97b admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:17.964220 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f0127-de39-48f8-bab9-0f9585935fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:17.964793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5df9482-31d2-48cd-8bd5-a215c38ea97b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:17.966777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5df9482-31d2-48cd-8bd5-a215c38ea97b admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:17.992134 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-cb1f0127-de39-48f8-bab9-0f9585935fb1 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:17.993522 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-cb1f0127-de39-48f8-bab9-0f9585935fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:18.035681 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 974/1953] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:48:17 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:48:18.037161 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df9482-31d2-48cd-8bd5-a215c38ea97b admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:18.043273 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-e5df9482-31d2-48cd-8bd5-a215c38ea97b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:18.104681 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 980/1954] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:48:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:48:18.105739 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f0127-de39-48f8-bab9-0f9585935fb1 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:18.110570 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-cb1f0127-de39-48f8-bab9-0f9585935fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:18.116331 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:18.138583 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f0127-de39-48f8-bab9-0f9585935fb1 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:18.138727 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df9482-31d2-48cd-8bd5-a215c38ea97b admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:18.144996 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a951c60b-12de-42ab-b269-3aa821a18a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:18.145688 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a951c60b-12de-42ab-b269-3aa821a18a78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:18.145997 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a951c60b-12de-42ab-b269-3aa821a18a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:18.146463 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a951c60b-12de-42ab-b269-3aa821a18a78 admin admin] PATH_INFO: `/v3/projects/36313c37d427415f8ba22a16f9b00994` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:18.146571 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ca2ff-79e8-4c25-b2ea-0cc766e57e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:18.147481 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ca2ff-79e8-4c25-b2ea-0cc766e57e5d admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:18.147670 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a951c60b-12de-42ab-b269-3aa821a18a78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36313c37d427415f8ba22a16f9b00994)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:18.147791 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ca2ff-79e8-4c25-b2ea-0cc766e57e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:18.148035 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ca2ff-79e8-4c25-b2ea-0cc766e57e5d admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:18.148675 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345ca2ff-79e8-4c25-b2ea-0cc766e57e5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:18.149970 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a951c60b-12de-42ab-b269-3aa821a18a78 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:18.150639 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345ca2ff-79e8-4c25-b2ea-0cc766e57e5d admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:18.183608 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 981/1955] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:48:18 2022] POST /identity/v3/users => generated 293 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:48:18.185936 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ca2ff-79e8-4c25-b2ea-0cc766e57e5d admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:18.191098 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-345ca2ff-79e8-4c25-b2ea-0cc766e57e5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:18.237944 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-a951c60b-12de-42ab-b269-3aa821a18a78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36313c37d427415f8ba22a16f9b00994', 'request_id': 'req-a951c60b-12de-42ab-b269-3aa821a18a78'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:18.264400 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-a951c60b-12de-42ab-b269-3aa821a18a78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36313c37d427415f8ba22a16f9b00994', 'request_id': 'req-a951c60b-12de-42ab-b269-3aa821a18a78'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:18.271475 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-a951c60b-12de-42ab-b269-3aa821a18a78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36313c37d427415f8ba22a16f9b00994', 'request_id': 'req-a951c60b-12de-42ab-b269-3aa821a18a78'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:18.275593 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ca2ff-79e8-4c25-b2ea-0cc766e57e5d admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:18.291649 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-a951c60b-12de-42ab-b269-3aa821a18a78 admin admin] The token cache is being invalidate because project 36313c37d427415f8ba22a16f9b00994 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:48:18.291940 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-a951c60b-12de-42ab-b269-3aa821a18a78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:18.294791 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-a951c60b-12de-42ab-b269-3aa821a18a78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:18.298279 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 975/1956] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:48:18 2022] DELETE /identity/v3/projects/36313c37d427415f8ba22a16f9b00994 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:48:18.299226 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a951c60b-12de-42ab-b269-3aa821a18a78 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:18.303248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-a951c60b-12de-42ab-b269-3aa821a18a78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:18.334350 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:18.359906 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f80bff-a2f3-414b-a8bd-3c07c3f325c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:18.360539 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f80bff-a2f3-414b-a8bd-3c07c3f325c7 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:18.360825 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f80bff-a2f3-414b-a8bd-3c07c3f325c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:18.361151 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f80bff-a2f3-414b-a8bd-3c07c3f325c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:18.364350 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-26f80bff-a2f3-414b-a8bd-3c07c3f325c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:18.436164 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a951c60b-12de-42ab-b269-3aa821a18a78 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:18.445770 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7886b41d-8f09-49e0-a0fc-d62bdad2f7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:18.446761 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886b41d-8f09-49e0-a0fc-d62bdad2f7a9 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:18.447927 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886b41d-8f09-49e0-a0fc-d62bdad2f7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:18.448362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886b41d-8f09-49e0-a0fc-d62bdad2f7a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:18.449471 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7886b41d-8f09-49e0-a0fc-d62bdad2f7a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:18.452233 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7886b41d-8f09-49e0-a0fc-d62bdad2f7a9 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:18.453242 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f80bff-a2f3-414b-a8bd-3c07c3f325c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:18.455391 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f80bff-a2f3-414b-a8bd-3c07c3f325c7 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:18.469398 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 982/1957] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:48:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:48:18.470929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f1808-387b-45b8-ace3-e40e9622e5fc None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:18.471240 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f1808-387b-45b8-ace3-e40e9622e5fc None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:18.471503 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f1808-387b-45b8-ace3-e40e9622e5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:18.477071 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 976/1958] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:48:18 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:48:18.481084 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7886b41d-8f09-49e0-a0fc-d62bdad2f7a9 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:18.495021 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-e82f1808-387b-45b8-ace3-e40e9622e5fc None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:18.496500 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-e82f1808-387b-45b8-ace3-e40e9622e5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:18.497939 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7886b41d-8f09-49e0-a0fc-d62bdad2f7a9 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:18.503888 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207c2ad-bcfe-46e4-a420-35034348ac76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:18.504808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207c2ad-bcfe-46e4-a420-35034348ac76 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:18.505273 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207c2ad-bcfe-46e4-a420-35034348ac76 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:18.506586 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 983/1959] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:48:18.507347 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207c2ad-bcfe-46e4-a420-35034348ac76 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:18.508586 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0207c2ad-bcfe-46e4-a420-35034348ac76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:18.510371 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0207c2ad-bcfe-46e4-a420-35034348ac76 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:18.514646 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f10bc-4da8-4143-a285-e97841ec7afe None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:18.515221 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f10bc-4da8-4143-a285-e97841ec7afe None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:18.515681 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f10bc-4da8-4143-a285-e97841ec7afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:18.525439 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 977/1960] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:48:18 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 22:48:18.529453 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207c2ad-bcfe-46e4-a420-35034348ac76 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:18.539255 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-3b9f10bc-4da8-4143-a285-e97841ec7afe None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:18.540858 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-3b9f10bc-4da8-4143-a285-e97841ec7afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:18.550162 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 984/1961] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:48:18.551204 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207c2ad-bcfe-46e4-a420-35034348ac76 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:18.556277 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f10bc-4da8-4143-a285-e97841ec7afe None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:18.558579 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df1cd0-b35f-4aa9-9011-a2b7af977e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:18.559438 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df1cd0-b35f-4aa9-9011-a2b7af977e7d admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:18.559872 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df1cd0-b35f-4aa9-9011-a2b7af977e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:18.560288 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df1cd0-b35f-4aa9-9011-a2b7af977e7d admin admin] PATH_INFO: `/v3/projects/1635e870725e400a9b25ad038f585a45/users/69015f7d89e54563b56ca8b0d8d91504/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:18.575527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f10bc-4da8-4143-a285-e97841ec7afe None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:18.582307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b293e6-6f8c-4e7f-aa25-3e4e81ea7d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:18.582932 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92df1cd0-b35f-4aa9-9011-a2b7af977e7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1635e870725e400a9b25ad038f585a45, user_id=69015f7d89e54563b56ca8b0d8d91504, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:18.583162 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b293e6-6f8c-4e7f-aa25-3e4e81ea7d8d admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:18.584095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b293e6-6f8c-4e7f-aa25-3e4e81ea7d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:18.584551 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b293e6-6f8c-4e7f-aa25-3e4e81ea7d8d admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:18.586479 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b293e6-6f8c-4e7f-aa25-3e4e81ea7d8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:48:18.586903 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b293e6-6f8c-4e7f-aa25-3e4e81ea7d8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:18.588023 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92df1cd0-b35f-4aa9-9011-a2b7af977e7d admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:18.591998 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b293e6-6f8c-4e7f-aa25-3e4e81ea7d8d admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:18.614441 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 985/1962] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:48:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:48:18.614692 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 978/1963] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:48:18 2022] PUT /identity/v3/projects/1635e870725e400a9b25ad038f585a45/users/69015f7d89e54563b56ca8b0d8d91504/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:48:18.615879 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b293e6-6f8c-4e7f-aa25-3e4e81ea7d8d admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:18.626996 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdcdc7-e8bc-4fdf-ba97-89f9de10be18 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:18.627824 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdcdc7-e8bc-4fdf-ba97-89f9de10be18 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:18.628186 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b293e6-6f8c-4e7f-aa25-3e4e81ea7d8d admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:18.628582 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdcdc7-e8bc-4fdf-ba97-89f9de10be18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:18.669494 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-a8cdcdc7-e8bc-4fdf-ba97-89f9de10be18 None None] MFA Rules not processed for user `69015f7d89e54563b56ca8b0d8d91504`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:18.671499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-a8cdcdc7-e8bc-4fdf-ba97-89f9de10be18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:18.721630 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c949038c-bbfa-4425-b08a-60191ba79e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:18.722395 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949038c-bbfa-4425-b08a-60191ba79e6d admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:18.722777 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949038c-bbfa-4425-b08a-60191ba79e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:18.723736 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949038c-bbfa-4425-b08a-60191ba79e6d admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:18.726055 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c949038c-bbfa-4425-b08a-60191ba79e6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:18.729264 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c949038c-bbfa-4425-b08a-60191ba79e6d admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:18.760340 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:18.801595 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 986/1964] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:48:18 2022] POST /identity/v3/projects => generated 371 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:48:18.804592 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c949038c-bbfa-4425-b08a-60191ba79e6d admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:18.836516 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 979/1965] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:48:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:48:18.852751 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cdcdc7-e8bc-4fdf-ba97-89f9de10be18 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:18.855134 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-a8cdcdc7-e8bc-4fdf-ba97-89f9de10be18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:18.905388 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:18.967523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cdcdc7-e8bc-4fdf-ba97-89f9de10be18 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:18.970795 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c949038c-bbfa-4425-b08a-60191ba79e6d admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:18.979045 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d38c0ec-c56a-4b9f-bf28-bb5d2ed4c985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:18.979812 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38c0ec-c56a-4b9f-bf28-bb5d2ed4c985 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:18.980199 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38c0ec-c56a-4b9f-bf28-bb5d2ed4c985 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:18.980727 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38c0ec-c56a-4b9f-bf28-bb5d2ed4c985 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:18.981662 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d38c0ec-c56a-4b9f-bf28-bb5d2ed4c985 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:18.984637 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d38c0ec-c56a-4b9f-bf28-bb5d2ed4c985 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:19.023673 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 987/1966] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:48:18 2022] POST /identity/v3/users => generated 286 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:48:19.034826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d38c0ec-c56a-4b9f-bf28-bb5d2ed4c985 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:19.064881 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2dfcc3-d625-4c9a-b9b0-9e37a9be0212 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:19.065882 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2dfcc3-d625-4c9a-b9b0-9e37a9be0212 service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:19.066076 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d38c0ec-c56a-4b9f-bf28-bb5d2ed4c985 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:19.066879 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2dfcc3-d625-4c9a-b9b0-9e37a9be0212 service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:19.067528 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2dfcc3-d625-4c9a-b9b0-9e37a9be0212 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:19.073037 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be836105-c78c-48f3-8741-a5f170fa5d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:19.073694 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be836105-c78c-48f3-8741-a5f170fa5d78 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:19.073992 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be836105-c78c-48f3-8741-a5f170fa5d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:19.074255 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be836105-c78c-48f3-8741-a5f170fa5d78 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:19.075127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be836105-c78c-48f3-8741-a5f170fa5d78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:19.077457 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be836105-c78c-48f3-8741-a5f170fa5d78 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:19.095995 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 988/1967] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:48:19 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:48:19.098010 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be836105-c78c-48f3-8741-a5f170fa5d78 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:19.113821 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be836105-c78c-48f3-8741-a5f170fa5d78 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:19.120093 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9be9d-2b8b-4db0-a4dc-fc6041c51dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:19.120997 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9be9d-2b8b-4db0-a4dc-fc6041c51dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:19.121526 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9be9d-2b8b-4db0-a4dc-fc6041c51dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:19.122362 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9be9d-2b8b-4db0-a4dc-fc6041c51dc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:19.123116 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9be9d-2b8b-4db0-a4dc-fc6041c51dc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:19.127957 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9be9d-2b8b-4db0-a4dc-fc6041c51dc1 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:19.146689 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 989/1968] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:48:19 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:48:19.148182 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9be9d-2b8b-4db0-a4dc-fc6041c51dc1 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:19.160330 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9be9d-2b8b-4db0-a4dc-fc6041c51dc1 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:19.168096 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504992b8-9714-442d-81a4-1e700bd4db95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:19.169063 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504992b8-9714-442d-81a4-1e700bd4db95 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:19.169501 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504992b8-9714-442d-81a4-1e700bd4db95 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:19.169981 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504992b8-9714-442d-81a4-1e700bd4db95 admin admin] PATH_INFO: `/v3/projects/50a0ee776f574f46bdb5f16c1b864402/users/0f645ba95eb64bb2b23d0d38a141307a/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:19.189245 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2dfcc3-d625-4c9a-b9b0-9e37a9be0212 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:19.191038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2dfcc3-d625-4c9a-b9b0-9e37a9be0212 service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:19.195490 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504992b8-9714-442d-81a4-1e700bd4db95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50a0ee776f574f46bdb5f16c1b864402, user_id=0f645ba95eb64bb2b23d0d38a141307a, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:19.202660 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504992b8-9714-442d-81a4-1e700bd4db95 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:19.203401 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 980/1969] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:48:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:48:19.204516 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2dfcc3-d625-4c9a-b9b0-9e37a9be0212 service cinder] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:19.205959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-3f2dfcc3-d625-4c9a-b9b0-9e37a9be0212 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:19.225951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2dfcc3-d625-4c9a-b9b0-9e37a9be0212 service cinder] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:19.241154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faff2ac-998e-4779-b5cc-995d58d957dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:19.242305 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faff2ac-998e-4779-b5cc-995d58d957dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:19.242866 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faff2ac-998e-4779-b5cc-995d58d957dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:19.244344 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faff2ac-998e-4779-b5cc-995d58d957dc admin admin] PATH_INFO: `/v3/projects/427b44e6dcfe476e8a895ae972db0534` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:19.246547 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faff2ac-998e-4779-b5cc-995d58d957dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=427b44e6dcfe476e8a895ae972db0534)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:19.246691 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 990/1970] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:48:19 2022] PUT /identity/v3/projects/50a0ee776f574f46bdb5f16c1b864402/users/0f645ba95eb64bb2b23d0d38a141307a/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:48:19.247569 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504992b8-9714-442d-81a4-1e700bd4db95 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:19.250905 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faff2ac-998e-4779-b5cc-995d58d957dc admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:19.341954 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:19.366658 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-8faff2ac-998e-4779-b5cc-995d58d957dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '427b44e6dcfe476e8a895ae972db0534', 'request_id': 'req-8faff2ac-998e-4779-b5cc-995d58d957dc'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:19.382161 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504992b8-9714-442d-81a4-1e700bd4db95 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:19.385003 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-8faff2ac-998e-4779-b5cc-995d58d957dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '427b44e6dcfe476e8a895ae972db0534', 'request_id': 'req-8faff2ac-998e-4779-b5cc-995d58d957dc'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:19.399543 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-8faff2ac-998e-4779-b5cc-995d58d957dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '427b44e6dcfe476e8a895ae972db0534', 'request_id': 'req-8faff2ac-998e-4779-b5cc-995d58d957dc'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:19.424355 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-8faff2ac-998e-4779-b5cc-995d58d957dc admin admin] The token cache is being invalidate because project 427b44e6dcfe476e8a895ae972db0534 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:48:19.429349 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-8faff2ac-998e-4779-b5cc-995d58d957dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:19.431070 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-8faff2ac-998e-4779-b5cc-995d58d957dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:19.439641 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 981/1971] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:48:19 2022] DELETE /identity/v3/projects/427b44e6dcfe476e8a895ae972db0534 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:48:19.443618 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844bb7d-3b5f-401d-b816-659e4cd195a6 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:19.445133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844bb7d-3b5f-401d-b816-659e4cd195a6 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:19.445642 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844bb7d-3b5f-401d-b816-659e4cd195a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:19.493821 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-4844bb7d-3b5f-401d-b816-659e4cd195a6 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:19.496248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-4844bb7d-3b5f-401d-b816-659e4cd195a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:19.507307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f3e22-6910-412e-b319-38de4a8cc6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:19.507969 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f3e22-6910-412e-b319-38de4a8cc6d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:19.509096 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f3e22-6910-412e-b319-38de4a8cc6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:19.509351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f3e22-6910-412e-b319-38de4a8cc6d7 admin admin] PATH_INFO: `/v3/users/69015f7d89e54563b56ca8b0d8d91504` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:19.510563 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3f3e22-6910-412e-b319-38de4a8cc6d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69015f7d89e54563b56ca8b0d8d91504)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:19.512966 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3f3e22-6910-412e-b319-38de4a8cc6d7 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:19.515866 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 982/1972] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:48:19.519712 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4844bb7d-3b5f-401d-b816-659e4cd195a6 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:19.524918 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-4844bb7d-3b5f-401d-b816-659e4cd195a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:19.554714 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4844bb7d-3b5f-401d-b816-659e4cd195a6 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:19.562106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27dea1-3a67-4958-b06c-10003d616fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:19.562663 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27dea1-3a67-4958-b06c-10003d616fd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:19.562948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27dea1-3a67-4958-b06c-10003d616fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:19.563532 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27dea1-3a67-4958-b06c-10003d616fd7 admin admin] PATH_INFO: `/v3/users/4dc8239481df43e1b95f8d5613a4b253` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:19.570964 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd27dea1-3a67-4958-b06c-10003d616fd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dc8239481df43e1b95f8d5613a4b253)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:19.572951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd27dea1-3a67-4958-b06c-10003d616fd7 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:19.591951 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:19.665046 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-5f3f3e22-6910-412e-b319-38de4a8cc6d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69015f7d89e54563b56ca8b0d8d91504'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:19.672699 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-fd27dea1-3a67-4958-b06c-10003d616fd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dc8239481df43e1b95f8d5613a4b253'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:19.686536 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-5f3f3e22-6910-412e-b319-38de4a8cc6d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69015f7d89e54563b56ca8b0d8d91504'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:19.688195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-fd27dea1-3a67-4958-b06c-10003d616fd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dc8239481df43e1b95f8d5613a4b253'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:19.698364 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-5f3f3e22-6910-412e-b319-38de4a8cc6d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69015f7d89e54563b56ca8b0d8d91504'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:19.702140 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-5f3f3e22-6910-412e-b319-38de4a8cc6d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69015f7d89e54563b56ca8b0d8d91504'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:19.702888 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-5f3f3e22-6910-412e-b319-38de4a8cc6d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69015f7d89e54563b56ca8b0d8d91504'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:19.731458 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-fd27dea1-3a67-4958-b06c-10003d616fd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dc8239481df43e1b95f8d5613a4b253'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:19.732234 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-fd27dea1-3a67-4958-b06c-10003d616fd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dc8239481df43e1b95f8d5613a4b253'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:19.732894 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-fd27dea1-3a67-4958-b06c-10003d616fd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dc8239481df43e1b95f8d5613a4b253'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:19.747097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 991/1973] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:48:19 2022] DELETE /identity/v3/users/69015f7d89e54563b56ca8b0d8d91504 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:48:19.750795 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f3e22-6910-412e-b319-38de4a8cc6d7 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:19.754481 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-5f3f3e22-6910-412e-b319-38de4a8cc6d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:19.765333 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 983/1974] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:48:19 2022] DELETE /identity/v3/users/4dc8239481df43e1b95f8d5613a4b253 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:48:19.765895 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4f28d-a344-429a-ad61-a3a39392015c None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:19.766434 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4f28d-a344-429a-ad61-a3a39392015c None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:19.767325 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4f28d-a344-429a-ad61-a3a39392015c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:19.814060 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-62c4f28d-a344-429a-ad61-a3a39392015c None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:19.815512 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-62c4f28d-a344-429a-ad61-a3a39392015c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:19.871301 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f3e22-6910-412e-b319-38de4a8cc6d7 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:19.892900 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 984/1975] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:48:19.904217 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4f28d-a344-429a-ad61-a3a39392015c None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:19.905961 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-62c4f28d-a344-429a-ad61-a3a39392015c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:19.942649 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e85b9f-2817-492e-b108-a76b5299debc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:19.944455 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e85b9f-2817-492e-b108-a76b5299debc service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:19.944455 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e85b9f-2817-492e-b108-a76b5299debc service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:19.945098 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e85b9f-2817-492e-b108-a76b5299debc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:19.947058 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-40e85b9f-2817-492e-b108-a76b5299debc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:19.960237 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:19.962300 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e85b9f-2817-492e-b108-a76b5299debc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:19.964042 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e85b9f-2817-492e-b108-a76b5299debc service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:19.980718 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 992/1976] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:48:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:48:20.018088 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4f28d-a344-429a-ad61-a3a39392015c None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:20.023047 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e46d5a-a31a-487b-9568-12c9b0ff3d29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:20.023746 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e46d5a-a31a-487b-9568-12c9b0ff3d29 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:20.024096 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e46d5a-a31a-487b-9568-12c9b0ff3d29 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:20.024902 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e46d5a-a31a-487b-9568-12c9b0ff3d29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:20.035342 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e46d5a-a31a-487b-9568-12c9b0ff3d29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:20.036727 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e46d5a-a31a-487b-9568-12c9b0ff3d29 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:20.049369 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 985/1977] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:48:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 22:48:20.283135 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e85b9f-2817-492e-b108-a76b5299debc service neutron] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:20.284563 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-40e85b9f-2817-492e-b108-a76b5299debc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:20.297780 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e85b9f-2817-492e-b108-a76b5299debc service neutron] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:20.303941 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bcdadd-843d-4282-a925-58c56abb6f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:20.304602 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bcdadd-843d-4282-a925-58c56abb6f2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:20.305185 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bcdadd-843d-4282-a925-58c56abb6f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:20.305185 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bcdadd-843d-4282-a925-58c56abb6f2f admin admin] PATH_INFO: `/v3/projects/35341b5eaded4998a70c7d5a844358b5` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:20.306463 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bcdadd-843d-4282-a925-58c56abb6f2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=35341b5eaded4998a70c7d5a844358b5)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:20.308399 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bcdadd-843d-4282-a925-58c56abb6f2f admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:20.361241 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-88bcdadd-843d-4282-a925-58c56abb6f2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35341b5eaded4998a70c7d5a844358b5', 'request_id': 'req-88bcdadd-843d-4282-a925-58c56abb6f2f'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:20.369415 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-88bcdadd-843d-4282-a925-58c56abb6f2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35341b5eaded4998a70c7d5a844358b5', 'request_id': 'req-88bcdadd-843d-4282-a925-58c56abb6f2f'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:20.373245 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:20.378130 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-88bcdadd-843d-4282-a925-58c56abb6f2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35341b5eaded4998a70c7d5a844358b5', 'request_id': 'req-88bcdadd-843d-4282-a925-58c56abb6f2f'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:20.393433 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-88bcdadd-843d-4282-a925-58c56abb6f2f admin admin] The token cache is being invalidate because project 35341b5eaded4998a70c7d5a844358b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:48:20.394378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-88bcdadd-843d-4282-a925-58c56abb6f2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:20.395418 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-88bcdadd-843d-4282-a925-58c56abb6f2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:20.397617 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 993/1978] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:48:20 2022] DELETE /identity/v3/projects/35341b5eaded4998a70c7d5a844358b5 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:48:20.402535 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfba810-e61e-4913-bf8c-5237c9a96287 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:20.402992 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfba810-e61e-4913-bf8c-5237c9a96287 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:20.403302 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfba810-e61e-4913-bf8c-5237c9a96287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:20.425795 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-adfba810-e61e-4913-bf8c-5237c9a96287 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:20.427200 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-adfba810-e61e-4913-bf8c-5237c9a96287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:20.492842 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 986/1979] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:48:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:48:20.500784 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7aca6-c975-4934-80d3-c6787c8d5733 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:20.501061 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7aca6-c975-4934-80d3-c6787c8d5733 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:20.501361 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7aca6-c975-4934-80d3-c6787c8d5733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:20.521843 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-36d7aca6-c975-4934-80d3-c6787c8d5733 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:20.523122 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-36d7aca6-c975-4934-80d3-c6787c8d5733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:20.531261 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 994/1980] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:48:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:48:20.535828 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfba810-e61e-4913-bf8c-5237c9a96287 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:20.547274 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfba810-e61e-4913-bf8c-5237c9a96287 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:20.553075 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e718d93-563b-4de2-8706-374de7534080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:20.553708 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e718d93-563b-4de2-8706-374de7534080 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:20.553989 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e718d93-563b-4de2-8706-374de7534080 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:20.554258 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e718d93-563b-4de2-8706-374de7534080 admin admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:20.554920 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e718d93-563b-4de2-8706-374de7534080 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:48:20.555222 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e718d93-563b-4de2-8706-374de7534080 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:20.556853 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e718d93-563b-4de2-8706-374de7534080 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:20.570764 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 987/1981] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:48:20.575524 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7aca6-c975-4934-80d3-c6787c8d5733 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:20.585384 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7aca6-c975-4934-80d3-c6787c8d5733 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:20.590135 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df79ed61-2dab-4adf-8d51-381fbb7037d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:20.590764 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df79ed61-2dab-4adf-8d51-381fbb7037d4 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:20.591039 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df79ed61-2dab-4adf-8d51-381fbb7037d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:20.591307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df79ed61-2dab-4adf-8d51-381fbb7037d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:20.591952 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df79ed61-2dab-4adf-8d51-381fbb7037d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:20.593932 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df79ed61-2dab-4adf-8d51-381fbb7037d4 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:20.624430 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 995/1982] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:48:20 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:48:20.628800 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e718d93-563b-4de2-8706-374de7534080 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:20.639998 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e718d93-563b-4de2-8706-374de7534080 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:20.705205 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dfe9e5-b7c8-453e-8121-6e6a6186a2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:20.706118 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dfe9e5-b7c8-453e-8121-6e6a6186a2fc admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:20.706706 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dfe9e5-b7c8-453e-8121-6e6a6186a2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:20.707153 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dfe9e5-b7c8-453e-8121-6e6a6186a2fc admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:20.708006 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dfe9e5-b7c8-453e-8121-6e6a6186a2fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:20.710918 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dfe9e5-b7c8-453e-8121-6e6a6186a2fc admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:20.752944 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 988/1983] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:48:20 2022] POST /identity/v3/users => generated 291 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:48:20.754393 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df79ed61-2dab-4adf-8d51-381fbb7037d4 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:20.767514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df79ed61-2dab-4adf-8d51-381fbb7037d4 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:20.775618 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6389b96-ed79-4666-8c28-e2c576c603c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:20.776861 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6389b96-ed79-4666-8c28-e2c576c603c4 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:20.776861 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6389b96-ed79-4666-8c28-e2c576c603c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:20.777055 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6389b96-ed79-4666-8c28-e2c576c603c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:20.777849 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6389b96-ed79-4666-8c28-e2c576c603c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:20.779646 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6389b96-ed79-4666-8c28-e2c576c603c4 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:20.798021 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 996/1984] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:48:20.799579 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dfe9e5-b7c8-453e-8121-6e6a6186a2fc admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:20.813476 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dfe9e5-b7c8-453e-8121-6e6a6186a2fc admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:20.823549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791734b-8be2-449d-9bc2-bef0a0338728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:20.824207 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791734b-8be2-449d-9bc2-bef0a0338728 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:20.824527 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791734b-8be2-449d-9bc2-bef0a0338728 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:20.824808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791734b-8be2-449d-9bc2-bef0a0338728 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:20.825500 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791734b-8be2-449d-9bc2-bef0a0338728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:20.827133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791734b-8be2-449d-9bc2-bef0a0338728 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:20.857434 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 989/1985] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 22:48:20.861318 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6389b96-ed79-4666-8c28-e2c576c603c4 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:20.880386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6389b96-ed79-4666-8c28-e2c576c603c4 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:20.885572 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574a343-deb3-4404-8d06-745c5e01b920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:20.886249 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574a343-deb3-4404-8d06-745c5e01b920 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:20.886535 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574a343-deb3-4404-8d06-745c5e01b920 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:20.886814 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574a343-deb3-4404-8d06-745c5e01b920 admin admin] PATH_INFO: `/v3/projects/c38d0effe2b242bbb7ea221ec2da0177/users/0f3c2f531a5942e6afabb16521523754/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:20.906263 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e574a343-deb3-4404-8d06-745c5e01b920 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c38d0effe2b242bbb7ea221ec2da0177, user_id=0f3c2f531a5942e6afabb16521523754, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:20.909077 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e574a343-deb3-4404-8d06-745c5e01b920 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:20.947454 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 997/1986] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:48:20 2022] PUT /identity/v3/projects/c38d0effe2b242bbb7ea221ec2da0177/users/0f3c2f531a5942e6afabb16521523754/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:48:20.952332 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea44e9-c16b-47ad-8177-36a66b4e0577 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:20.952620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea44e9-c16b-47ad-8177-36a66b4e0577 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:20.953019 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea44e9-c16b-47ad-8177-36a66b4e0577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:21.000425 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-4aea44e9-c16b-47ad-8177-36a66b4e0577 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:21.007004 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-4aea44e9-c16b-47ad-8177-36a66b4e0577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:21.132865 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:21.211470 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 990/1987] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:48:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:48:21.230860 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574a343-deb3-4404-8d06-745c5e01b920 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:21.232989 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-e574a343-deb3-4404-8d06-745c5e01b920 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:21.342936 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574a343-deb3-4404-8d06-745c5e01b920 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:21.350096 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb066e-7ebb-4319-9477-b6cedc536283 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:21.350981 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb066e-7ebb-4319-9477-b6cedc536283 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:21.351671 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb066e-7ebb-4319-9477-b6cedc536283 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:21.352106 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb066e-7ebb-4319-9477-b6cedc536283 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:21.369057 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcb066e-7ebb-4319-9477-b6cedc536283 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:21.371691 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcb066e-7ebb-4319-9477-b6cedc536283 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:21.393120 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 998/1988] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:48:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:48:21.544171 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea44e9-c16b-47ad-8177-36a66b4e0577 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:21.547450 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-4aea44e9-c16b-47ad-8177-36a66b4e0577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:21.585086 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea44e9-c16b-47ad-8177-36a66b4e0577 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:21.620675 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3878d4b-4443-4b52-885a-a9f5b0221fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:21.621270 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3878d4b-4443-4b52-885a-a9f5b0221fb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:21.621649 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3878d4b-4443-4b52-885a-a9f5b0221fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:21.622483 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3878d4b-4443-4b52-885a-a9f5b0221fb0 admin admin] PATH_INFO: `/v3/projects/1635e870725e400a9b25ad038f585a45` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:21.624962 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3878d4b-4443-4b52-885a-a9f5b0221fb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1635e870725e400a9b25ad038f585a45)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:21.627048 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3878d4b-4443-4b52-885a-a9f5b0221fb0 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:21.700268 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f3878d4b-4443-4b52-885a-a9f5b0221fb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1635e870725e400a9b25ad038f585a45', 'request_id': 'req-f3878d4b-4443-4b52-885a-a9f5b0221fb0'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:21.711111 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f3878d4b-4443-4b52-885a-a9f5b0221fb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1635e870725e400a9b25ad038f585a45', 'request_id': 'req-f3878d4b-4443-4b52-885a-a9f5b0221fb0'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:21.723545 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f3878d4b-4443-4b52-885a-a9f5b0221fb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1635e870725e400a9b25ad038f585a45', 'request_id': 'req-f3878d4b-4443-4b52-885a-a9f5b0221fb0'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:21.739435 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f3878d4b-4443-4b52-885a-a9f5b0221fb0 admin admin] The token cache is being invalidate because project 1635e870725e400a9b25ad038f585a45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:48:21.740768 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f3878d4b-4443-4b52-885a-a9f5b0221fb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:21.741841 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f3878d4b-4443-4b52-885a-a9f5b0221fb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:48:21.744772 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 991/1989] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:48:21 2022] DELETE /identity/v3/projects/1635e870725e400a9b25ad038f585a45 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:48:21.751791 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356e166-31e5-4ff8-8506-0fe5aa34a285 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:21.752361 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356e166-31e5-4ff8-8506-0fe5aa34a285 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:21.752457 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356e166-31e5-4ff8-8506-0fe5aa34a285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:21.776628 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-9356e166-31e5-4ff8-8506-0fe5aa34a285 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:21.778017 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-9356e166-31e5-4ff8-8506-0fe5aa34a285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:21.898458 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 999/1990] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:48:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:48:21.907363 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb68032-c00e-49c1-ac6b-1a0da4602247 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:21.907655 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb68032-c00e-49c1-ac6b-1a0da4602247 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:21.907893 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb68032-c00e-49c1-ac6b-1a0da4602247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:21.932821 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-beb68032-c00e-49c1-ac6b-1a0da4602247 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:21.934153 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-beb68032-c00e-49c1-ac6b-1a0da4602247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:21.945607 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 992/1991] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:48:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:48:21.950200 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9356e166-31e5-4ff8-8506-0fe5aa34a285 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:21.963438 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9356e166-31e5-4ff8-8506-0fe5aa34a285 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:21.988693 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adfe154-1d48-4abb-ba39-e1ed744d28b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:21.989358 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfe154-1d48-4abb-ba39-e1ed744d28b9 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:21.989713 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfe154-1d48-4abb-ba39-e1ed744d28b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:21.989982 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfe154-1d48-4abb-ba39-e1ed744d28b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:21.991139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adfe154-1d48-4abb-ba39-e1ed744d28b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:48:21.991139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adfe154-1d48-4abb-ba39-e1ed744d28b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:21.992745 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adfe154-1d48-4abb-ba39-e1ed744d28b9 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:22.010559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1000/1992] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:48:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:48:22.011749 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb68032-c00e-49c1-ac6b-1a0da4602247 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:22.022314 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb68032-c00e-49c1-ac6b-1a0da4602247 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:22.028217 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ad6f1-31f1-4715-ad7e-a09ac23cb40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:22.029220 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ad6f1-31f1-4715-ad7e-a09ac23cb40d admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:22.029603 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ad6f1-31f1-4715-ad7e-a09ac23cb40d admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:22.029945 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ad6f1-31f1-4715-ad7e-a09ac23cb40d admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:22.030710 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5ad6f1-31f1-4715-ad7e-a09ac23cb40d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:22.033090 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5ad6f1-31f1-4715-ad7e-a09ac23cb40d admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:22.077222 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 993/1993] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:48:22 2022] POST /identity/v3/projects => generated 369 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:48:22.081311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adfe154-1d48-4abb-ba39-e1ed744d28b9 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:22.094794 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adfe154-1d48-4abb-ba39-e1ed744d28b9 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:22.128113 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:22.193244 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b9fce-f0ed-4f69-b0d1-d60893e228c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:22.193681 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b9fce-f0ed-4f69-b0d1-d60893e228c6 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:22.194033 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b9fce-f0ed-4f69-b0d1-d60893e228c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:22.194377 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b9fce-f0ed-4f69-b0d1-d60893e228c6 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:22.195073 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b9fce-f0ed-4f69-b0d1-d60893e228c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:22.198698 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b9fce-f0ed-4f69-b0d1-d60893e228c6 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:22.223689 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1001/1994] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:48:22 2022] POST /identity/v3/users => generated 285 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:48:22.229395 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ad6f1-31f1-4715-ad7e-a09ac23cb40d admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:22.248563 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ad6f1-31f1-4715-ad7e-a09ac23cb40d admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:22.258154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd65cca-077b-4100-b0ff-45e94ecf4f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:22.259213 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd65cca-077b-4100-b0ff-45e94ecf4f5f admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:22.259987 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd65cca-077b-4100-b0ff-45e94ecf4f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:22.260328 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd65cca-077b-4100-b0ff-45e94ecf4f5f admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:22.261577 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd65cca-077b-4100-b0ff-45e94ecf4f5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:22.264683 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd65cca-077b-4100-b0ff-45e94ecf4f5f admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:22.285976 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 994/1995] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:48:22.291836 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b9fce-f0ed-4f69-b0d1-d60893e228c6 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:22.318627 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b9fce-f0ed-4f69-b0d1-d60893e228c6 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:22.328035 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49305220-7e82-47cd-b5ea-f337d9775e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:22.329195 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49305220-7e82-47cd-b5ea-f337d9775e34 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:22.329922 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49305220-7e82-47cd-b5ea-f337d9775e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:22.330606 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49305220-7e82-47cd-b5ea-f337d9775e34 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:22.332064 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49305220-7e82-47cd-b5ea-f337d9775e34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:22.336003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49305220-7e82-47cd-b5ea-f337d9775e34 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:22.364430 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1002/1996] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:48:22.371369 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd65cca-077b-4100-b0ff-45e94ecf4f5f admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:22.395400 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd65cca-077b-4100-b0ff-45e94ecf4f5f admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:22.407791 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0deaee4-65a1-4641-ab84-505eebe9557c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:22.409196 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0deaee4-65a1-4641-ab84-505eebe9557c admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:22.409860 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0deaee4-65a1-4641-ab84-505eebe9557c admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:22.410398 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0deaee4-65a1-4641-ab84-505eebe9557c admin admin] PATH_INFO: `/v3/projects/8434b921ce3c477195275a74d140d8b7/users/6f487a36d01f43c6824e261320d76c84/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:22.463558 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0deaee4-65a1-4641-ab84-505eebe9557c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8434b921ce3c477195275a74d140d8b7, user_id=6f487a36d01f43c6824e261320d76c84, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:22.468431 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0deaee4-65a1-4641-ab84-505eebe9557c admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:22.487879 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 995/1997] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:48:22 2022] PUT /identity/v3/projects/8434b921ce3c477195275a74d140d8b7/users/6f487a36d01f43c6824e261320d76c84/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:48:22.498902 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7350cf-78bd-45d1-ab15-525eea4b5cba None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:22.499259 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7350cf-78bd-45d1-ab15-525eea4b5cba None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:22.499567 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7350cf-78bd-45d1-ab15-525eea4b5cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:22.557373 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddddb5e-70b6-4f7e-95ec-6416ef6c6633 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:22.557842 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddddb5e-70b6-4f7e-95ec-6416ef6c6633 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:22.557939 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddddb5e-70b6-4f7e-95ec-6416ef6c6633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:22.570527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-fb7350cf-78bd-45d1-ab15-525eea4b5cba None None] MFA Rules not processed for user `6f487a36d01f43c6824e261320d76c84`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:22.572715 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-fb7350cf-78bd-45d1-ab15-525eea4b5cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:22.636606 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-3ddddb5e-70b6-4f7e-95ec-6416ef6c6633 None None] MFA Rules not processed for user `0f645ba95eb64bb2b23d0d38a141307a`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:22.638109 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-3ddddb5e-70b6-4f7e-95ec-6416ef6c6633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:22.706850 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1003/1998] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:48:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:48:22.731412 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7350cf-78bd-45d1-ab15-525eea4b5cba None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:22.736433 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-fb7350cf-78bd-45d1-ab15-525eea4b5cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:22.769214 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:22.829385 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 996/1999] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:48:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:48:22.875921 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15265d45-2534-4384-a3c9-27dc9ee6e057 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:22.877074 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15265d45-2534-4384-a3c9-27dc9ee6e057 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:22.877610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15265d45-2534-4384-a3c9-27dc9ee6e057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:22.918546 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7350cf-78bd-45d1-ab15-525eea4b5cba None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:22.921521 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-15265d45-2534-4384-a3c9-27dc9ee6e057 None None] MFA Rules not processed for user `0f3c2f531a5942e6afabb16521523754`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:22.923794 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-15265d45-2534-4384-a3c9-27dc9ee6e057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:22.925041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bb6c5-96e9-4b16-8261-c8375a452d13 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc7524edf5d5422f91d92fd91be5ee1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:22.926648 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bb6c5-96e9-4b16-8261-c8375a452d13 service glance] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:22.927100 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bb6c5-96e9-4b16-8261-c8375a452d13 service glance] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:22.928030 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bb6c5-96e9-4b16-8261-c8375a452d13 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:22.942184 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4bb6c5-96e9-4b16-8261-c8375a452d13 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:22.945198 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4bb6c5-96e9-4b16-8261-c8375a452d13 service glance] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:22.957432 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1004/2000] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:48:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:48:22.958504 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bb6c5-96e9-4b16-8261-c8375a452d13 service glance] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:22.959878 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-0b4bb6c5-96e9-4b16-8261-c8375a452d13 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:22.972073 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:23.082625 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 997/2001] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:48:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:48:23.088717 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bb6c5-96e9-4b16-8261-c8375a452d13 service glance] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:23.106774 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb1114-04d7-4731-a33a-c6044e378900 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:23.107710 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb1114-04d7-4731-a33a-c6044e378900 service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:23.108231 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb1114-04d7-4731-a33a-c6044e378900 service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:23.108665 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb1114-04d7-4731-a33a-c6044e378900 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:23.121518 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcb1114-04d7-4731-a33a-c6044e378900 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:23.123700 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcb1114-04d7-4731-a33a-c6044e378900 service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:23.137087 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1005/2002] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:48:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:48:23.577013 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15265d45-2534-4384-a3c9-27dc9ee6e057 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:48:23.590951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15265d45-2534-4384-a3c9-27dc9ee6e057 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:48:23.594827 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26565cd4-b613-46a8-976a-c53f96a82f51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:48:23.595482 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26565cd4-b613-46a8-976a-c53f96a82f51 service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:23.595756 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26565cd4-b613-46a8-976a-c53f96a82f51 service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:23.596028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26565cd4-b613-46a8-976a-c53f96a82f51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:23.606634 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26565cd4-b613-46a8-976a-c53f96a82f51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:48:23.608352 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26565cd4-b613-46a8-976a-c53f96a82f51 service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:48:23.621011 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 998/2003] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:48:23 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:48:24.780126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3cd2d6-560a-4d1c-bf3e-bdeb0c9afe94 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:24.780126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3cd2d6-560a-4d1c-bf3e-bdeb0c9afe94 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:24.780764 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3cd2d6-560a-4d1c-bf3e-bdeb0c9afe94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:24.821789 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-7b3cd2d6-560a-4d1c-bf3e-bdeb0c9afe94 None None] MFA Rules not processed for user `0f645ba95eb64bb2b23d0d38a141307a`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:24.823761 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-7b3cd2d6-560a-4d1c-bf3e-bdeb0c9afe94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:24.833738 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1006/2004] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:48:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:48:25.068597 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ecf54-561c-4b3c-9665-ec2e77be766e None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:48:25.069856 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ecf54-561c-4b3c-9665-ec2e77be766e None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:48:25.070137 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ecf54-561c-4b3c-9665-ec2e77be766e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:48:25.104970 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-624ecf54-561c-4b3c-9665-ec2e77be766e None None] MFA Rules not processed for user `0f3c2f531a5942e6afabb16521523754`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:48:25.106668 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-624ecf54-561c-4b3c-9665-ec2e77be766e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:48:25.121668 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 999/2005] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:48:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:48:34.812872 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=25 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:48:35.096695 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=46 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:05.087920 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3cd2d6-560a-4d1c-bf3e-bdeb0c9afe94 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:05.089597 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-7b3cd2d6-560a-4d1c-bf3e-bdeb0c9afe94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:05.157687 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3cd2d6-560a-4d1c-bf3e-bdeb0c9afe94 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:05.162396 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93823738-b5ed-4404-b8e7-c6227683e79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:05.163033 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93823738-b5ed-4404-b8e7-c6227683e79a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:05.163307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93823738-b5ed-4404-b8e7-c6227683e79a admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:05.163582 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93823738-b5ed-4404-b8e7-c6227683e79a admin admin] PATH_INFO: `/v3/users/c2d0bf2ba7274336bd201872eea4d2c8` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:05.164698 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93823738-b5ed-4404-b8e7-c6227683e79a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2d0bf2ba7274336bd201872eea4d2c8)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:05.166663 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93823738-b5ed-4404-b8e7-c6227683e79a admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:05.220826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-93823738-b5ed-4404-b8e7-c6227683e79a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2d0bf2ba7274336bd201872eea4d2c8'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:05.229606 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-93823738-b5ed-4404-b8e7-c6227683e79a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2d0bf2ba7274336bd201872eea4d2c8'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:05.236782 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-93823738-b5ed-4404-b8e7-c6227683e79a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2d0bf2ba7274336bd201872eea4d2c8'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:05.237514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-93823738-b5ed-4404-b8e7-c6227683e79a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2d0bf2ba7274336bd201872eea4d2c8'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:05.238148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-93823738-b5ed-4404-b8e7-c6227683e79a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2d0bf2ba7274336bd201872eea4d2c8'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:05.255310 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1007/2006] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:49:05 2022] DELETE /identity/v3/users/c2d0bf2ba7274336bd201872eea4d2c8 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:49:05.259755 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f059940-c3d1-445e-87f4-23da44a16a3e None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:05.260354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f059940-c3d1-445e-87f4-23da44a16a3e None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:05.260354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f059940-c3d1-445e-87f4-23da44a16a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:05.279149 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-3f059940-c3d1-445e-87f4-23da44a16a3e None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:05.280291 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-3f059940-c3d1-445e-87f4-23da44a16a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:05.338315 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1000/2007] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:49:05.349061 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93823738-b5ed-4404-b8e7-c6227683e79a admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:05.350545 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-93823738-b5ed-4404-b8e7-c6227683e79a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:05.407232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93823738-b5ed-4404-b8e7-c6227683e79a admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:05.411329 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83041fe5-7229-48d5-b7c2-6b2836670a1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:05.411935 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83041fe5-7229-48d5-b7c2-6b2836670a1b service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:05.412218 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83041fe5-7229-48d5-b7c2-6b2836670a1b service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:05.412493 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83041fe5-7229-48d5-b7c2-6b2836670a1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:05.421801 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83041fe5-7229-48d5-b7c2-6b2836670a1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:05.423278 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83041fe5-7229-48d5-b7c2-6b2836670a1b service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:05.433659 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1008/2008] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:49:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:49:05.738637 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f059940-c3d1-445e-87f4-23da44a16a3e None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:05.740161 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-3f059940-c3d1-445e-87f4-23da44a16a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:05.761978 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f059940-c3d1-445e-87f4-23da44a16a3e None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:05.767763 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470a5ab-35d7-47f2-9f82-9254d8c65c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:05.769628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470a5ab-35d7-47f2-9f82-9254d8c65c5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:05.770212 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470a5ab-35d7-47f2-9f82-9254d8c65c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:05.770498 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470a5ab-35d7-47f2-9f82-9254d8c65c5f admin admin] PATH_INFO: `/v3/projects/1545b3fa31274b879c523897476eee09` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:05.772014 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3470a5ab-35d7-47f2-9f82-9254d8c65c5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1545b3fa31274b879c523897476eee09)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:05.773818 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3470a5ab-35d7-47f2-9f82-9254d8c65c5f admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:05.830854 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-3470a5ab-35d7-47f2-9f82-9254d8c65c5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1545b3fa31274b879c523897476eee09', 'request_id': 'req-3470a5ab-35d7-47f2-9f82-9254d8c65c5f'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:05.837641 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-3470a5ab-35d7-47f2-9f82-9254d8c65c5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1545b3fa31274b879c523897476eee09', 'request_id': 'req-3470a5ab-35d7-47f2-9f82-9254d8c65c5f'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:05.843612 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-3470a5ab-35d7-47f2-9f82-9254d8c65c5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1545b3fa31274b879c523897476eee09', 'request_id': 'req-3470a5ab-35d7-47f2-9f82-9254d8c65c5f'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:05.854960 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-3470a5ab-35d7-47f2-9f82-9254d8c65c5f admin admin] The token cache is being invalidate because project 1545b3fa31274b879c523897476eee09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:49:05.855246 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-3470a5ab-35d7-47f2-9f82-9254d8c65c5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:05.855888 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-3470a5ab-35d7-47f2-9f82-9254d8c65c5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:05.857414 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1001/2009] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:49:05 2022] DELETE /identity/v3/projects/1545b3fa31274b879c523897476eee09 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:49:05.861457 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83041fe5-7229-48d5-b7c2-6b2836670a1b service neutron] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:05.871898 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83041fe5-7229-48d5-b7c2-6b2836670a1b service neutron] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:05.890413 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:05.932354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e31915-8365-4750-9aa2-0dc957f89dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:05.932984 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e31915-8365-4750-9aa2-0dc957f89dc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:05.933260 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e31915-8365-4750-9aa2-0dc957f89dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:05.933557 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e31915-8365-4750-9aa2-0dc957f89dc2 admin admin] PATH_INFO: `/v3/users/b57da6d2359044dc8ae0be1ec2fbe7e8` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:05.934666 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e31915-8365-4750-9aa2-0dc957f89dc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b57da6d2359044dc8ae0be1ec2fbe7e8)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:05.936618 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e31915-8365-4750-9aa2-0dc957f89dc2 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:05.982196 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-c5e31915-8365-4750-9aa2-0dc957f89dc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b57da6d2359044dc8ae0be1ec2fbe7e8'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:05.989869 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-c5e31915-8365-4750-9aa2-0dc957f89dc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b57da6d2359044dc8ae0be1ec2fbe7e8'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:05.996782 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-c5e31915-8365-4750-9aa2-0dc957f89dc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b57da6d2359044dc8ae0be1ec2fbe7e8'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:05.997461 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-c5e31915-8365-4750-9aa2-0dc957f89dc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b57da6d2359044dc8ae0be1ec2fbe7e8'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:05.998077 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-c5e31915-8365-4750-9aa2-0dc957f89dc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b57da6d2359044dc8ae0be1ec2fbe7e8'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:06.015564 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1009/2010] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:49:05 2022] DELETE /identity/v3/users/b57da6d2359044dc8ae0be1ec2fbe7e8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:49:06.720687 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470a5ab-35d7-47f2-9f82-9254d8c65c5f admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:06.722317 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-3470a5ab-35d7-47f2-9f82-9254d8c65c5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:06.740082 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470a5ab-35d7-47f2-9f82-9254d8c65c5f admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:06.863879 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f50fbc0-bf69-484c-b4ac-2af68c08ce8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:06.865210 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50fbc0-bf69-484c-b4ac-2af68c08ce8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:06.865210 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50fbc0-bf69-484c-b4ac-2af68c08ce8f admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:06.865770 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50fbc0-bf69-484c-b4ac-2af68c08ce8f admin admin] PATH_INFO: `/v3/projects/c1821fb9d4fe4610a61e465663b00261` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:06.871145 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f50fbc0-bf69-484c-b4ac-2af68c08ce8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1821fb9d4fe4610a61e465663b00261)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:06.873926 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f50fbc0-bf69-484c-b4ac-2af68c08ce8f admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:06.919298 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:06.947628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-2f50fbc0-bf69-484c-b4ac-2af68c08ce8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1821fb9d4fe4610a61e465663b00261', 'request_id': 'req-2f50fbc0-bf69-484c-b4ac-2af68c08ce8f'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:06.960049 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-2f50fbc0-bf69-484c-b4ac-2af68c08ce8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1821fb9d4fe4610a61e465663b00261', 'request_id': 'req-2f50fbc0-bf69-484c-b4ac-2af68c08ce8f'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:06.980054 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-2f50fbc0-bf69-484c-b4ac-2af68c08ce8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1821fb9d4fe4610a61e465663b00261', 'request_id': 'req-2f50fbc0-bf69-484c-b4ac-2af68c08ce8f'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:06.995193 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-2f50fbc0-bf69-484c-b4ac-2af68c08ce8f admin admin] The token cache is being invalidate because project c1821fb9d4fe4610a61e465663b00261 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:49:06.995646 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-2f50fbc0-bf69-484c-b4ac-2af68c08ce8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:06.996457 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-2f50fbc0-bf69-484c-b4ac-2af68c08ce8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:07.001326 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1002/2011] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:49:06 2022] DELETE /identity/v3/projects/c1821fb9d4fe4610a61e465663b00261 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:49:07.005900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84ef5e-a90f-4383-8600-a700442aa0b3 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:07.006379 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84ef5e-a90f-4383-8600-a700442aa0b3 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:07.006469 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84ef5e-a90f-4383-8600-a700442aa0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:07.036268 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-9f84ef5e-a90f-4383-8600-a700442aa0b3 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:07.037642 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-9f84ef5e-a90f-4383-8600-a700442aa0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:07.133425 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bdf41-f03d-4da0-a702-460b01054400 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:07.133788 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1010/2012] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:49:07.134029 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bdf41-f03d-4da0-a702-460b01054400 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:07.134325 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bdf41-f03d-4da0-a702-460b01054400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:07.169456 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-cc2bdf41-f03d-4da0-a702-460b01054400 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:07.172223 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-cc2bdf41-f03d-4da0-a702-460b01054400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:07.190663 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f84ef5e-a90f-4383-8600-a700442aa0b3 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:07.191262 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1003/2013] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:49:07.204463 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f84ef5e-a90f-4383-8600-a700442aa0b3 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:07.210443 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1bfcae-03ea-4a3e-a93b-2706af91af93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:07.211212 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1bfcae-03ea-4a3e-a93b-2706af91af93 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:07.211563 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1bfcae-03ea-4a3e-a93b-2706af91af93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:07.211905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1bfcae-03ea-4a3e-a93b-2706af91af93 admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:07.212633 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1bfcae-03ea-4a3e-a93b-2706af91af93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:49:07.213313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1bfcae-03ea-4a3e-a93b-2706af91af93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:07.214837 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1bfcae-03ea-4a3e-a93b-2706af91af93 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:07.229395 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1011/2014] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:49:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:49:07.233428 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2bdf41-f03d-4da0-a702-460b01054400 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:07.246094 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2bdf41-f03d-4da0-a702-460b01054400 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:07.261314 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ed9c8-4757-482d-8dc6-5419ca1f2c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:07.261314 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ed9c8-4757-482d-8dc6-5419ca1f2c7b admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:07.261314 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ed9c8-4757-482d-8dc6-5419ca1f2c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:07.261314 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ed9c8-4757-482d-8dc6-5419ca1f2c7b admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:07.261742 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5ed9c8-4757-482d-8dc6-5419ca1f2c7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:07.261742 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5ed9c8-4757-482d-8dc6-5419ca1f2c7b admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:07.301388 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1004/2015] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:49:07 2022] POST /identity/v3/projects => generated 355 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:49:07.305393 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1bfcae-03ea-4a3e-a93b-2706af91af93 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:07.327990 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1bfcae-03ea-4a3e-a93b-2706af91af93 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:07.366219 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:07.402994 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd18ed-0c7a-4ae8-83a4-479d789edddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:07.403625 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd18ed-0c7a-4ae8-83a4-479d789edddc admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:07.403930 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd18ed-0c7a-4ae8-83a4-479d789edddc admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:07.404193 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd18ed-0c7a-4ae8-83a4-479d789edddc admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:07.404825 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fd18ed-0c7a-4ae8-83a4-479d789edddc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:07.407206 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fd18ed-0c7a-4ae8-83a4-479d789edddc admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:07.444169 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1012/2016] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:49:07 2022] POST /identity/v3/users => generated 278 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:49:07.448528 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ed9c8-4757-482d-8dc6-5419ca1f2c7b admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:07.461143 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ed9c8-4757-482d-8dc6-5419ca1f2c7b admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:07.468112 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b310f-928d-4f4f-af84-d990f3b75a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:07.468757 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b310f-928d-4f4f-af84-d990f3b75a19 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:07.469314 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b310f-928d-4f4f-af84-d990f3b75a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:07.469314 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b310f-928d-4f4f-af84-d990f3b75a19 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:07.469982 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b310f-928d-4f4f-af84-d990f3b75a19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:07.471769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b310f-928d-4f4f-af84-d990f3b75a19 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:07.489204 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1005/2017] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:49:07.491045 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd18ed-0c7a-4ae8-83a4-479d789edddc admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:07.504342 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd18ed-0c7a-4ae8-83a4-479d789edddc admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:07.521146 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bf539c-2396-4312-8550-864b2b8218e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:07.521146 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf539c-2396-4312-8550-864b2b8218e3 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:07.521146 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf539c-2396-4312-8550-864b2b8218e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:07.521146 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf539c-2396-4312-8550-864b2b8218e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:07.521492 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bf539c-2396-4312-8550-864b2b8218e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:07.521492 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bf539c-2396-4312-8550-864b2b8218e3 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:07.534284 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1013/2018] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:49:07.535830 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b310f-928d-4f4f-af84-d990f3b75a19 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:07.549090 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b310f-928d-4f4f-af84-d990f3b75a19 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:07.560151 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6bc045-2ff4-4ac0-8284-dd419c742c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:07.561235 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6bc045-2ff4-4ac0-8284-dd419c742c0f admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:07.561781 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6bc045-2ff4-4ac0-8284-dd419c742c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:07.562281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6bc045-2ff4-4ac0-8284-dd419c742c0f admin admin] PATH_INFO: `/v3/projects/3c7b513237474dd9b7854741a9176e2c/users/a44c70192f6647d29206f8b5f77953ab/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:07.589253 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6bc045-2ff4-4ac0-8284-dd419c742c0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c7b513237474dd9b7854741a9176e2c, user_id=a44c70192f6647d29206f8b5f77953ab, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:07.594673 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6bc045-2ff4-4ac0-8284-dd419c742c0f admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:07.616135 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1006/2019] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:49:07 2022] PUT /identity/v3/projects/3c7b513237474dd9b7854741a9176e2c/users/a44c70192f6647d29206f8b5f77953ab/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:49:07.626260 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431087ea-cd91-460a-a6be-3449a0d13f92 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:07.626595 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431087ea-cd91-460a-a6be-3449a0d13f92 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:07.626850 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431087ea-cd91-460a-a6be-3449a0d13f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:07.666541 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-431087ea-cd91-460a-a6be-3449a0d13f92 None None] MFA Rules not processed for user `a44c70192f6647d29206f8b5f77953ab`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:07.667817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-431087ea-cd91-460a-a6be-3449a0d13f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:07.779684 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1014/2020] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:07 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:49:07.785739 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6bc045-2ff4-4ac0-8284-dd419c742c0f admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:07.866743 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6bc045-2ff4-4ac0-8284-dd419c742c0f admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:07.874968 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4faa222-81f8-4813-aafd-b1cf0ca49daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:07.875599 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4faa222-81f8-4813-aafd-b1cf0ca49daf admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:07.875881 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4faa222-81f8-4813-aafd-b1cf0ca49daf admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:07.876158 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4faa222-81f8-4813-aafd-b1cf0ca49daf admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:07.876878 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4faa222-81f8-4813-aafd-b1cf0ca49daf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:07.882347 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4faa222-81f8-4813-aafd-b1cf0ca49daf admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:07.927583 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1007/2021] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:49:07 2022] POST /identity/v3/projects => generated 355 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:49:07.929076 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431087ea-cd91-460a-a6be-3449a0d13f92 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:07.941523 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431087ea-cd91-460a-a6be-3449a0d13f92 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:08.010625 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dbf575-3af1-4811-84e3-235dba93ff3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:08.012053 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dbf575-3af1-4811-84e3-235dba93ff3a admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:08.012408 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dbf575-3af1-4811-84e3-235dba93ff3a admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:08.012714 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dbf575-3af1-4811-84e3-235dba93ff3a admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:08.013470 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dbf575-3af1-4811-84e3-235dba93ff3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:08.015492 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dbf575-3af1-4811-84e3-235dba93ff3a admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:08.040838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1015/2022] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:49:07 2022] POST /identity/v3/users => generated 284 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:49:08.044626 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4faa222-81f8-4813-aafd-b1cf0ca49daf admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:08.063383 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4faa222-81f8-4813-aafd-b1cf0ca49daf admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:08.071177 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1004aa95-f38e-4216-a45f-44806e65b750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:08.071806 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004aa95-f38e-4216-a45f-44806e65b750 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:08.072087 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004aa95-f38e-4216-a45f-44806e65b750 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:08.072388 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004aa95-f38e-4216-a45f-44806e65b750 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:08.073007 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1004aa95-f38e-4216-a45f-44806e65b750 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:08.074689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1004aa95-f38e-4216-a45f-44806e65b750 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:08.090196 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1008/2023] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:49:08 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:49:08.091247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dbf575-3af1-4811-84e3-235dba93ff3a admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:08.108701 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dbf575-3af1-4811-84e3-235dba93ff3a admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:08.108701 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d2c8ba-73cb-4959-8129-9f8b0cf17acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:08.108701 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2c8ba-73cb-4959-8129-9f8b0cf17acb admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:08.108701 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2c8ba-73cb-4959-8129-9f8b0cf17acb admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:08.109347 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2c8ba-73cb-4959-8129-9f8b0cf17acb admin admin] PATH_INFO: `/v3/projects/b3e4320d05314707b85180f214009c18/users/754e963325404349b364125c4e73b508/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:08.125184 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d2c8ba-73cb-4959-8129-9f8b0cf17acb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3e4320d05314707b85180f214009c18, user_id=754e963325404349b364125c4e73b508, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:08.128846 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d2c8ba-73cb-4959-8129-9f8b0cf17acb admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:08.144105 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1016/2024] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:49:08 2022] PUT /identity/v3/projects/b3e4320d05314707b85180f214009c18/users/754e963325404349b364125c4e73b508/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:49:08.148318 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1004aa95-f38e-4216-a45f-44806e65b750 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:08.158323 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1004aa95-f38e-4216-a45f-44806e65b750 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:08.180172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:08.225000 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d18449-191c-4aa3-81ca-5d0aaf4d8045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:08.225656 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d18449-191c-4aa3-81ca-5d0aaf4d8045 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:08.225933 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d18449-191c-4aa3-81ca-5d0aaf4d8045 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:08.226203 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d18449-191c-4aa3-81ca-5d0aaf4d8045 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:08.226925 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d18449-191c-4aa3-81ca-5d0aaf4d8045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:08.228525 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d18449-191c-4aa3-81ca-5d0aaf4d8045 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:08.238334 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1009/2025] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:49:08 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:49:08.242676 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d2c8ba-73cb-4959-8129-9f8b0cf17acb admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:08.248166 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:08.254012 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d2c8ba-73cb-4959-8129-9f8b0cf17acb admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:08.259472 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0fc481-88ea-4360-8937-613abdbdcff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:08.260160 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fc481-88ea-4360-8937-613abdbdcff2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:08.260470 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fc481-88ea-4360-8937-613abdbdcff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:08.260771 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fc481-88ea-4360-8937-613abdbdcff2 admin admin] PATH_INFO: `/v3/projects/b3e4320d05314707b85180f214009c18/users/754e963325404349b364125c4e73b508/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:08.262835 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0fc481-88ea-4360-8937-613abdbdcff2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3e4320d05314707b85180f214009c18, user_id=754e963325404349b364125c4e73b508, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:08.265526 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0fc481-88ea-4360-8937-613abdbdcff2 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:08.277135 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1017/2026] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:49:08 2022] PUT /identity/v3/projects/b3e4320d05314707b85180f214009c18/users/754e963325404349b364125c4e73b508/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:49:08.293918 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab4a6e-205f-4a25-ace3-aad01d492545 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:08.294204 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab4a6e-205f-4a25-ace3-aad01d492545 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:08.294477 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab4a6e-205f-4a25-ace3-aad01d492545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:08.321895 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-dbab4a6e-205f-4a25-ace3-aad01d492545 None None] MFA Rules not processed for user `754e963325404349b364125c4e73b508`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:08.323110 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-dbab4a6e-205f-4a25-ace3-aad01d492545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:08.396885 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1010/2027] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:08 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:49:08.408616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0fc481-88ea-4360-8937-613abdbdcff2 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:08.411091 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-5a0fc481-88ea-4360-8937-613abdbdcff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:08.474200 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0fc481-88ea-4360-8937-613abdbdcff2 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:08.478078 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5045bb-dc05-4bec-8e87-305f40e6a7eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:08.478785 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5045bb-dc05-4bec-8e87-305f40e6a7eb service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:08.479062 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5045bb-dc05-4bec-8e87-305f40e6a7eb service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:08.479339 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5045bb-dc05-4bec-8e87-305f40e6a7eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:08.490121 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5045bb-dc05-4bec-8e87-305f40e6a7eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:08.491582 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5045bb-dc05-4bec-8e87-305f40e6a7eb service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:08.502121 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1018/2028] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:49:08 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:49:08.591011 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab4a6e-205f-4a25-ace3-aad01d492545 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:08.602856 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab4a6e-205f-4a25-ace3-aad01d492545 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:08.606842 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed0fc5-5148-4efb-8708-01afdf50512c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:08.608614 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed0fc5-5148-4efb-8708-01afdf50512c service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:08.609087 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed0fc5-5148-4efb-8708-01afdf50512c service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:08.609657 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed0fc5-5148-4efb-8708-01afdf50512c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:08.672665 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ed0fc5-5148-4efb-8708-01afdf50512c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:08.674148 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ed0fc5-5148-4efb-8708-01afdf50512c service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:08.684628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1011/2029] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:49:08 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:49:18.497346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:18.680356 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=80 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:21.607992 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5045bb-dc05-4bec-8e87-305f40e6a7eb service cinder] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:21.609456 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-bc5045bb-dc05-4bec-8e87-305f40e6a7eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:21.675426 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5045bb-dc05-4bec-8e87-305f40e6a7eb service cinder] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:21.680230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3212a4-775d-4750-8157-5d8474600f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:21.680854 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3212a4-775d-4750-8157-5d8474600f1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:21.681149 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3212a4-775d-4750-8157-5d8474600f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:21.681517 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3212a4-775d-4750-8157-5d8474600f1f admin admin] PATH_INFO: `/v3/users/6f487a36d01f43c6824e261320d76c84` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:21.682644 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3212a4-775d-4750-8157-5d8474600f1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f487a36d01f43c6824e261320d76c84)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:21.684577 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3212a4-775d-4750-8157-5d8474600f1f admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:21.733706 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-9c3212a4-775d-4750-8157-5d8474600f1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f487a36d01f43c6824e261320d76c84'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:21.742292 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-9c3212a4-775d-4750-8157-5d8474600f1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f487a36d01f43c6824e261320d76c84'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:21.749289 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-9c3212a4-775d-4750-8157-5d8474600f1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f487a36d01f43c6824e261320d76c84'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:21.750007 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-9c3212a4-775d-4750-8157-5d8474600f1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f487a36d01f43c6824e261320d76c84'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:21.750673 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-9c3212a4-775d-4750-8157-5d8474600f1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f487a36d01f43c6824e261320d76c84'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:21.771572 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1019/2030] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:49:21 2022] DELETE /identity/v3/users/6f487a36d01f43c6824e261320d76c84 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:49:21.776243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724a5d6-7012-4af1-869b-c8b8c9815ec7 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:21.776741 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724a5d6-7012-4af1-869b-c8b8c9815ec7 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:21.776833 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724a5d6-7012-4af1-869b-c8b8c9815ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:21.795865 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-3724a5d6-7012-4af1-869b-c8b8c9815ec7 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:21.797078 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-3724a5d6-7012-4af1-869b-c8b8c9815ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:21.855796 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1012/2031] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:49:21.866597 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3212a4-775d-4750-8157-5d8474600f1f admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:21.868019 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-9c3212a4-775d-4750-8157-5d8474600f1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:21.927248 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3212a4-775d-4750-8157-5d8474600f1f admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:21.931544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29bf35-c323-4454-a0ec-37b78eb581ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:21.932163 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29bf35-c323-4454-a0ec-37b78eb581ab service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:21.932442 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29bf35-c323-4454-a0ec-37b78eb581ab service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:21.932714 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29bf35-c323-4454-a0ec-37b78eb581ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:21.942098 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29bf35-c323-4454-a0ec-37b78eb581ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:21.943529 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29bf35-c323-4454-a0ec-37b78eb581ab service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:21.952799 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1020/2032] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:49:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 22:49:22.237674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3724a5d6-7012-4af1-869b-c8b8c9815ec7 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:22.239235 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-3724a5d6-7012-4af1-869b-c8b8c9815ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:22.256054 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3724a5d6-7012-4af1-869b-c8b8c9815ec7 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:22.261427 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ac94e-dc94-41ae-acf9-e701797bc181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:22.262208 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ac94e-dc94-41ae-acf9-e701797bc181 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:22.262687 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ac94e-dc94-41ae-acf9-e701797bc181 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:22.263023 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ac94e-dc94-41ae-acf9-e701797bc181 admin admin] PATH_INFO: `/v3/projects/8434b921ce3c477195275a74d140d8b7` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:22.277542 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11ac94e-dc94-41ae-acf9-e701797bc181 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8434b921ce3c477195275a74d140d8b7)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:22.281435 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11ac94e-dc94-41ae-acf9-e701797bc181 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:22.374598 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-b11ac94e-dc94-41ae-acf9-e701797bc181 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8434b921ce3c477195275a74d140d8b7', 'request_id': 'req-b11ac94e-dc94-41ae-acf9-e701797bc181'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:22.389635 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-b11ac94e-dc94-41ae-acf9-e701797bc181 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8434b921ce3c477195275a74d140d8b7', 'request_id': 'req-b11ac94e-dc94-41ae-acf9-e701797bc181'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:22.409266 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-b11ac94e-dc94-41ae-acf9-e701797bc181 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8434b921ce3c477195275a74d140d8b7', 'request_id': 'req-b11ac94e-dc94-41ae-acf9-e701797bc181'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:22.430069 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-b11ac94e-dc94-41ae-acf9-e701797bc181 admin admin] The token cache is being invalidate because project 8434b921ce3c477195275a74d140d8b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:49:22.433173 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-b11ac94e-dc94-41ae-acf9-e701797bc181 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:22.433617 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-b11ac94e-dc94-41ae-acf9-e701797bc181 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:22.435285 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1013/2033] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:49:22 2022] DELETE /identity/v3/projects/8434b921ce3c477195275a74d140d8b7 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:49:22.441411 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6508b-8b87-488f-93c7-81f49ebf6d55 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:22.441704 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6508b-8b87-488f-93c7-81f49ebf6d55 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:22.442030 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6508b-8b87-488f-93c7-81f49ebf6d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:22.467034 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-8fa6508b-8b87-488f-93c7-81f49ebf6d55 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:22.469357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-8fa6508b-8b87-488f-93c7-81f49ebf6d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:22.493009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:22.572952 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e38ea-6391-42fc-99a0-7faf42ecd17d None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:22.573241 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e38ea-6391-42fc-99a0-7faf42ecd17d None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:22.573598 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e38ea-6391-42fc-99a0-7faf42ecd17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:22.584515 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1021/2034] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:49:22.599883 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-8d9e38ea-6391-42fc-99a0-7faf42ecd17d None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:22.602356 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-8d9e38ea-6391-42fc-99a0-7faf42ecd17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:22.623513 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1014/2035] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:49:22.628220 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa6508b-8b87-488f-93c7-81f49ebf6d55 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:22.646082 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa6508b-8b87-488f-93c7-81f49ebf6d55 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:22.658197 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13a9122-dd75-4d7b-8cf3-a16eeaacb9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:22.659664 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a9122-dd75-4d7b-8cf3-a16eeaacb9e8 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:22.659954 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a9122-dd75-4d7b-8cf3-a16eeaacb9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:22.660235 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a9122-dd75-4d7b-8cf3-a16eeaacb9e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:22.660936 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13a9122-dd75-4d7b-8cf3-a16eeaacb9e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:49:22.661243 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13a9122-dd75-4d7b-8cf3-a16eeaacb9e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:22.662915 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13a9122-dd75-4d7b-8cf3-a16eeaacb9e8 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:22.676666 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1022/2036] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:49:22.681216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e38ea-6391-42fc-99a0-7faf42ecd17d None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:22.696939 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e38ea-6391-42fc-99a0-7faf42ecd17d None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:22.708441 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a12b5-ef0c-4d07-a60b-1e97950a1a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:22.709585 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a12b5-ef0c-4d07-a60b-1e97950a1a82 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:22.710044 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a12b5-ef0c-4d07-a60b-1e97950a1a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:22.710395 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a12b5-ef0c-4d07-a60b-1e97950a1a82 admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:22.711208 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82a12b5-ef0c-4d07-a60b-1e97950a1a82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:22.713684 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82a12b5-ef0c-4d07-a60b-1e97950a1a82 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:22.747465 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1015/2037] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:49:22 2022] POST /identity/v3/projects => generated 373 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:49:22.752560 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13a9122-dd75-4d7b-8cf3-a16eeaacb9e8 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:22.764304 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13a9122-dd75-4d7b-8cf3-a16eeaacb9e8 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:22.883947 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f272dc-a8be-43a6-ab31-ad5f88c6f92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:22.884767 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f272dc-a8be-43a6-ab31-ad5f88c6f92c admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:22.884973 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f272dc-a8be-43a6-ab31-ad5f88c6f92c admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:22.885235 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f272dc-a8be-43a6-ab31-ad5f88c6f92c admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:22.885902 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f272dc-a8be-43a6-ab31-ad5f88c6f92c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:22.887873 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f272dc-a8be-43a6-ab31-ad5f88c6f92c admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:22.942056 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1023/2038] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:49:22 2022] POST /identity/v3/users => generated 287 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:49:22.943324 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a12b5-ef0c-4d07-a60b-1e97950a1a82 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:22.957810 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a12b5-ef0c-4d07-a60b-1e97950a1a82 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:22.970508 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c8bff-95fd-4b0a-9ff4-398f7a3540fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:22.971358 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c8bff-95fd-4b0a-9ff4-398f7a3540fb admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:22.971858 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c8bff-95fd-4b0a-9ff4-398f7a3540fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:22.972308 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c8bff-95fd-4b0a-9ff4-398f7a3540fb admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:22.973059 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c8bff-95fd-4b0a-9ff4-398f7a3540fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:22.975299 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c8bff-95fd-4b0a-9ff4-398f7a3540fb admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:23.013493 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1016/2039] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:49:23.018045 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f272dc-a8be-43a6-ab31-ad5f88c6f92c admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:23.035134 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f272dc-a8be-43a6-ab31-ad5f88c6f92c admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:23.042196 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725da43-a789-4f14-a465-f647c1a93f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:23.042826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725da43-a789-4f14-a465-f647c1a93f28 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:23.043371 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725da43-a789-4f14-a465-f647c1a93f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:23.043651 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725da43-a789-4f14-a465-f647c1a93f28 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:23.044306 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7725da43-a789-4f14-a465-f647c1a93f28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:23.046161 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7725da43-a789-4f14-a465-f647c1a93f28 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:23.061820 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1024/2040] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 22:49:23.065363 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c8bff-95fd-4b0a-9ff4-398f7a3540fb admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:23.081702 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c8bff-95fd-4b0a-9ff4-398f7a3540fb admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:23.087554 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d898c258-1edb-4dfa-897c-52ea7ea609b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:23.088692 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898c258-1edb-4dfa-897c-52ea7ea609b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:23.089078 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898c258-1edb-4dfa-897c-52ea7ea609b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:23.089449 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898c258-1edb-4dfa-897c-52ea7ea609b9 admin admin] PATH_INFO: `/v3/projects/7e881c1dc4534853852f5724c0f409e9/users/c743153e3ac04c8f8ff17c6401548938/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:23.105937 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d898c258-1edb-4dfa-897c-52ea7ea609b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e881c1dc4534853852f5724c0f409e9, user_id=c743153e3ac04c8f8ff17c6401548938, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:23.108638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d898c258-1edb-4dfa-897c-52ea7ea609b9 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:23.123286 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1017/2041] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:49:23 2022] PUT /identity/v3/projects/7e881c1dc4534853852f5724c0f409e9/users/c743153e3ac04c8f8ff17c6401548938/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:49:23.134721 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92e35c-df12-4f94-a96a-e90fa731787a None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:23.135011 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92e35c-df12-4f94-a96a-e90fa731787a None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:23.135317 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92e35c-df12-4f94-a96a-e90fa731787a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:23.167531 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-8f92e35c-df12-4f94-a96a-e90fa731787a None None] MFA Rules not processed for user `c743153e3ac04c8f8ff17c6401548938`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:23.168711 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-8f92e35c-df12-4f94-a96a-e90fa731787a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:23.238461 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1025/2042] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:49:23.249871 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d898c258-1edb-4dfa-897c-52ea7ea609b9 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:23.251589 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-d898c258-1edb-4dfa-897c-52ea7ea609b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:23.275107 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:23.316227 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d898c258-1edb-4dfa-897c-52ea7ea609b9 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:23.320351 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ccd685-dc23-459a-b731-c57ba590b3ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:23.321050 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ccd685-dc23-459a-b731-c57ba590b3ee service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:23.321443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ccd685-dc23-459a-b731-c57ba590b3ee service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:23.322430 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ccd685-dc23-459a-b731-c57ba590b3ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:23.332384 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ccd685-dc23-459a-b731-c57ba590b3ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:23.334083 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ccd685-dc23-459a-b731-c57ba590b3ee service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:23.344270 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1018/2043] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:49:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:49:23.369228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f92e35c-df12-4f94-a96a-e90fa731787a None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:23.392826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:23.446117 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f92e35c-df12-4f94-a96a-e90fa731787a None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:23.450793 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f8bf9-dbab-4c0e-bfb8-581580d4ecb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:23.451410 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f8bf9-dbab-4c0e-bfb8-581580d4ecb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:23.451685 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f8bf9-dbab-4c0e-bfb8-581580d4ecb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:23.451953 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f8bf9-dbab-4c0e-bfb8-581580d4ecb7 admin admin] PATH_INFO: `/v3/users/c743153e3ac04c8f8ff17c6401548938` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:23.453846 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f8bf9-dbab-4c0e-bfb8-581580d4ecb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c743153e3ac04c8f8ff17c6401548938)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:23.456967 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f8bf9-dbab-4c0e-bfb8-581580d4ecb7 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:23.514983 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-479f8bf9-dbab-4c0e-bfb8-581580d4ecb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c743153e3ac04c8f8ff17c6401548938'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:23.525332 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-479f8bf9-dbab-4c0e-bfb8-581580d4ecb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c743153e3ac04c8f8ff17c6401548938'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:23.544795 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-479f8bf9-dbab-4c0e-bfb8-581580d4ecb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c743153e3ac04c8f8ff17c6401548938'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:23.546438 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-479f8bf9-dbab-4c0e-bfb8-581580d4ecb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c743153e3ac04c8f8ff17c6401548938'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:23.547745 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-479f8bf9-dbab-4c0e-bfb8-581580d4ecb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c743153e3ac04c8f8ff17c6401548938'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:23.584552 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1026/2044] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:49:23 2022] DELETE /identity/v3/users/c743153e3ac04c8f8ff17c6401548938 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:49:23.589051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf45da4-008c-4bbb-be8d-f9d82984081e None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:23.589387 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf45da4-008c-4bbb-be8d-f9d82984081e None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:23.589670 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf45da4-008c-4bbb-be8d-f9d82984081e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:23.615319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-cdf45da4-008c-4bbb-be8d-f9d82984081e None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:23.616617 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-cdf45da4-008c-4bbb-be8d-f9d82984081e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:23.701649 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1019/2045] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:49:23.710178 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f8bf9-dbab-4c0e-bfb8-581580d4ecb7 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:23.711857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-479f8bf9-dbab-4c0e-bfb8-581580d4ecb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:23.783682 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f8bf9-dbab-4c0e-bfb8-581580d4ecb7 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:23.787463 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4a939-cadc-4fe2-a9b2-099662365316 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:23.788106 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4a939-cadc-4fe2-a9b2-099662365316 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:23.788390 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4a939-cadc-4fe2-a9b2-099662365316 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:23.788667 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4a939-cadc-4fe2-a9b2-099662365316 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:23.797803 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4a939-cadc-4fe2-a9b2-099662365316 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:23.799426 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4a939-cadc-4fe2-a9b2-099662365316 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:23.809580 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1027/2046] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:49:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 22:49:24.097633 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf45da4-008c-4bbb-be8d-f9d82984081e None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:24.099210 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-cdf45da4-008c-4bbb-be8d-f9d82984081e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:24.113070 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf45da4-008c-4bbb-be8d-f9d82984081e None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:24.118127 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d328ff85-d6e9-4bf3-b99d-9d86b29a0de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:24.118837 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328ff85-d6e9-4bf3-b99d-9d86b29a0de2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:24.119037 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328ff85-d6e9-4bf3-b99d-9d86b29a0de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:24.119313 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328ff85-d6e9-4bf3-b99d-9d86b29a0de2 admin admin] PATH_INFO: `/v3/projects/7e881c1dc4534853852f5724c0f409e9` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:24.120446 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d328ff85-d6e9-4bf3-b99d-9d86b29a0de2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e881c1dc4534853852f5724c0f409e9)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:24.122561 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d328ff85-d6e9-4bf3-b99d-9d86b29a0de2 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:24.180887 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-d328ff85-d6e9-4bf3-b99d-9d86b29a0de2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e881c1dc4534853852f5724c0f409e9', 'request_id': 'req-d328ff85-d6e9-4bf3-b99d-9d86b29a0de2'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:24.187889 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-d328ff85-d6e9-4bf3-b99d-9d86b29a0de2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e881c1dc4534853852f5724c0f409e9', 'request_id': 'req-d328ff85-d6e9-4bf3-b99d-9d86b29a0de2'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:24.193453 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-d328ff85-d6e9-4bf3-b99d-9d86b29a0de2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e881c1dc4534853852f5724c0f409e9', 'request_id': 'req-d328ff85-d6e9-4bf3-b99d-9d86b29a0de2'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:24.204915 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-d328ff85-d6e9-4bf3-b99d-9d86b29a0de2 admin admin] The token cache is being invalidate because project 7e881c1dc4534853852f5724c0f409e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:49:24.205210 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-d328ff85-d6e9-4bf3-b99d-9d86b29a0de2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:24.206126 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-d328ff85-d6e9-4bf3-b99d-9d86b29a0de2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:24.207618 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1020/2047] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:49:24 2022] DELETE /identity/v3/projects/7e881c1dc4534853852f5724c0f409e9 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:49:24.212524 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16486ce-d4ee-4a32-ae3f-dd9ea11a3ebe None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:24.212814 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16486ce-d4ee-4a32-ae3f-dd9ea11a3ebe None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:24.213094 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16486ce-d4ee-4a32-ae3f-dd9ea11a3ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:24.234244 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-f16486ce-d4ee-4a32-ae3f-dd9ea11a3ebe None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:24.235826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-f16486ce-d4ee-4a32-ae3f-dd9ea11a3ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:24.271636 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:24.300966 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1028/2048] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:49:24.310439 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcbc324-aa2a-4060-aa0f-8cff4972487d None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:24.310949 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcbc324-aa2a-4060-aa0f-8cff4972487d None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:24.311466 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcbc324-aa2a-4060-aa0f-8cff4972487d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:24.331930 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-dfcbc324-aa2a-4060-aa0f-8cff4972487d None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:24.333167 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-dfcbc324-aa2a-4060-aa0f-8cff4972487d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:24.341150 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1021/2049] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:49:24.346083 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16486ce-d4ee-4a32-ae3f-dd9ea11a3ebe None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:24.356889 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16486ce-d4ee-4a32-ae3f-dd9ea11a3ebe None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:24.362939 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f691382-7db7-47dd-81fe-dca44872b919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:24.363578 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f691382-7db7-47dd-81fe-dca44872b919 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:24.363840 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f691382-7db7-47dd-81fe-dca44872b919 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:24.364113 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f691382-7db7-47dd-81fe-dca44872b919 admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:24.364768 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f691382-7db7-47dd-81fe-dca44872b919 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:49:24.365068 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f691382-7db7-47dd-81fe-dca44872b919 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:24.366741 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f691382-7db7-47dd-81fe-dca44872b919 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:24.382941 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1029/2050] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:49:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:49:24.384992 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcbc324-aa2a-4060-aa0f-8cff4972487d None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:24.398466 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcbc324-aa2a-4060-aa0f-8cff4972487d None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:24.403334 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577b465d-68ea-46e6-abb5-3b5f45dd45bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:24.403944 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b465d-68ea-46e6-abb5-3b5f45dd45bb admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:24.404225 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b465d-68ea-46e6-abb5-3b5f45dd45bb admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:24.404500 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b465d-68ea-46e6-abb5-3b5f45dd45bb admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:24.405123 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577b465d-68ea-46e6-abb5-3b5f45dd45bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:24.407074 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577b465d-68ea-46e6-abb5-3b5f45dd45bb admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:24.440860 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1022/2051] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:49:24 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:49:24.443187 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f691382-7db7-47dd-81fe-dca44872b919 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:24.453584 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f691382-7db7-47dd-81fe-dca44872b919 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:24.510048 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af0b9c9-be61-4b07-ac9d-16ac16d97ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:24.510651 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0b9c9-be61-4b07-ac9d-16ac16d97ff7 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:24.510930 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0b9c9-be61-4b07-ac9d-16ac16d97ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:24.511301 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0b9c9-be61-4b07-ac9d-16ac16d97ff7 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:24.512069 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af0b9c9-be61-4b07-ac9d-16ac16d97ff7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:24.514092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af0b9c9-be61-4b07-ac9d-16ac16d97ff7 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:24.541523 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1030/2052] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:49:24 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:49:24.543693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577b465d-68ea-46e6-abb5-3b5f45dd45bb admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:24.554666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577b465d-68ea-46e6-abb5-3b5f45dd45bb admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:24.559443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a0f38-3c1c-42c3-abaa-e9599bbd6e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:24.560053 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a0f38-3c1c-42c3-abaa-e9599bbd6e1f admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:24.560339 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a0f38-3c1c-42c3-abaa-e9599bbd6e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:24.560639 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a0f38-3c1c-42c3-abaa-e9599bbd6e1f admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:24.561421 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a0f38-3c1c-42c3-abaa-e9599bbd6e1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:24.563232 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a0f38-3c1c-42c3-abaa-e9599bbd6e1f admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:24.573551 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1023/2053] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 22:49:24.577623 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af0b9c9-be61-4b07-ac9d-16ac16d97ff7 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:24.588638 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af0b9c9-be61-4b07-ac9d-16ac16d97ff7 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:24.593503 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc9c5df-2524-4bee-97e5-229d8b874bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:24.594102 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9c5df-2524-4bee-97e5-229d8b874bb9 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:24.594377 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9c5df-2524-4bee-97e5-229d8b874bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:24.594648 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9c5df-2524-4bee-97e5-229d8b874bb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:24.595923 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc9c5df-2524-4bee-97e5-229d8b874bb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:24.597603 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc9c5df-2524-4bee-97e5-229d8b874bb9 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:24.608070 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1031/2054] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:49:24.612256 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a0f38-3c1c-42c3-abaa-e9599bbd6e1f admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:24.622349 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a0f38-3c1c-42c3-abaa-e9599bbd6e1f admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:24.627154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a229b-da3e-47b5-ac07-526d0a59318f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:24.627806 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a229b-da3e-47b5-ac07-526d0a59318f admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:24.628081 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a229b-da3e-47b5-ac07-526d0a59318f admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:24.628351 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a229b-da3e-47b5-ac07-526d0a59318f admin admin] PATH_INFO: `/v3/projects/dcf2ffea147f4700b67bfc7accdd0815/users/e15f5f4d1ca74dba89d80dfa5ca87563/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:24.630764 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:24.647013 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a229b-da3e-47b5-ac07-526d0a59318f admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcf2ffea147f4700b67bfc7accdd0815, user_id=e15f5f4d1ca74dba89d80dfa5ca87563, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:24.650166 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a229b-da3e-47b5-ac07-526d0a59318f admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:24.664219 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1024/2055] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:49:24 2022] PUT /identity/v3/projects/dcf2ffea147f4700b67bfc7accdd0815/users/e15f5f4d1ca74dba89d80dfa5ca87563/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:49:24.676911 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ee7de-4185-40b8-80b8-7a8c97b30914 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:24.677382 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ee7de-4185-40b8-80b8-7a8c97b30914 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:24.677683 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ee7de-4185-40b8-80b8-7a8c97b30914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:24.707778 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-132ee7de-4185-40b8-80b8-7a8c97b30914 None None] MFA Rules not processed for user `e15f5f4d1ca74dba89d80dfa5ca87563`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:24.708916 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-132ee7de-4185-40b8-80b8-7a8c97b30914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:24.789751 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1032/2056] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:49:24.798172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a229b-da3e-47b5-ac07-526d0a59318f admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:24.803521 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-b02a229b-da3e-47b5-ac07-526d0a59318f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:24.873716 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a229b-da3e-47b5-ac07-526d0a59318f admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:24.879620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c395bb-061e-4477-b76c-112481d0c571 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:24.880421 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c395bb-061e-4477-b76c-112481d0c571 service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:24.881406 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c395bb-061e-4477-b76c-112481d0c571 service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:24.881406 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c395bb-061e-4477-b76c-112481d0c571 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:24.899238 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c395bb-061e-4477-b76c-112481d0c571 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:24.900697 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c395bb-061e-4477-b76c-112481d0c571 service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:24.910009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1025/2057] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:49:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:49:27.375960 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a73d27-538c-43db-a138-60c1049103d0 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:27.376717 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a73d27-538c-43db-a138-60c1049103d0 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:27.376717 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a73d27-538c-43db-a138-60c1049103d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:27.399201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-49a73d27-538c-43db-a138-60c1049103d0 None None] MFA Rules not processed for user `e15f5f4d1ca74dba89d80dfa5ca87563`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:27.400529 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-49a73d27-538c-43db-a138-60c1049103d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:27.410437 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1033/2058] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:49:29.383467 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c395bb-061e-4477-b76c-112481d0c571 service cinder] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:29.385885 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-53c395bb-061e-4477-b76c-112481d0c571 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:29.472909 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c395bb-061e-4477-b76c-112481d0c571 service cinder] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:29.475650 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd6ad3d-8c96-477b-b41e-ae8bb8008a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:29.476301 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6ad3d-8c96-477b-b41e-ae8bb8008a33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:29.476723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6ad3d-8c96-477b-b41e-ae8bb8008a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:29.477254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6ad3d-8c96-477b-b41e-ae8bb8008a33 admin admin] PATH_INFO: `/v3/users/0d4dc87fae144ab9a40e1f047330168d` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:29.478930 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd6ad3d-8c96-477b-b41e-ae8bb8008a33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d4dc87fae144ab9a40e1f047330168d)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:29.481705 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd6ad3d-8c96-477b-b41e-ae8bb8008a33 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:29.550672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-edd6ad3d-8c96-477b-b41e-ae8bb8008a33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d4dc87fae144ab9a40e1f047330168d'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:29.560346 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-edd6ad3d-8c96-477b-b41e-ae8bb8008a33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d4dc87fae144ab9a40e1f047330168d'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:29.586986 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-edd6ad3d-8c96-477b-b41e-ae8bb8008a33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d4dc87fae144ab9a40e1f047330168d'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:29.588068 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-edd6ad3d-8c96-477b-b41e-ae8bb8008a33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d4dc87fae144ab9a40e1f047330168d'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:29.588842 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-edd6ad3d-8c96-477b-b41e-ae8bb8008a33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d4dc87fae144ab9a40e1f047330168d'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:29.603100 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a73d27-538c-43db-a138-60c1049103d0 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:29.604774 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-49a73d27-538c-43db-a138-60c1049103d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:29.605447 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1026/2059] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:49:29 2022] DELETE /identity/v3/users/0d4dc87fae144ab9a40e1f047330168d => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:49:29.628803 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a73d27-538c-43db-a138-60c1049103d0 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:29.656646 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:29.691237 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1a505-ee82-403b-a69f-84794fab4bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:29.691893 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1a505-ee82-403b-a69f-84794fab4bee admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:29.692156 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1a505-ee82-403b-a69f-84794fab4bee admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:29.692431 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1a505-ee82-403b-a69f-84794fab4bee admin admin] PATH_INFO: `/v3/users/a44c70192f6647d29206f8b5f77953ab` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:29.693657 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd1a505-ee82-403b-a69f-84794fab4bee admin admin] RBAC: Authorizing `identity:delete_user(user_id=a44c70192f6647d29206f8b5f77953ab)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:29.695637 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd1a505-ee82-403b-a69f-84794fab4bee admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:29.748575 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-6bd1a505-ee82-403b-a69f-84794fab4bee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a44c70192f6647d29206f8b5f77953ab'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:29.758804 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-6bd1a505-ee82-403b-a69f-84794fab4bee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a44c70192f6647d29206f8b5f77953ab'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:29.767256 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-6bd1a505-ee82-403b-a69f-84794fab4bee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a44c70192f6647d29206f8b5f77953ab'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:29.768035 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-6bd1a505-ee82-403b-a69f-84794fab4bee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a44c70192f6647d29206f8b5f77953ab'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:29.768741 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-6bd1a505-ee82-403b-a69f-84794fab4bee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a44c70192f6647d29206f8b5f77953ab'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:29.788449 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1034/2060] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:49:29 2022] DELETE /identity/v3/users/a44c70192f6647d29206f8b5f77953ab => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:49:29.793460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1a505-ee82-403b-a69f-84794fab4bee admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:29.793807 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af1bd3-874e-42fc-b710-c189c0f26e43 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:29.794089 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af1bd3-874e-42fc-b710-c189c0f26e43 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:29.794447 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af1bd3-874e-42fc-b710-c189c0f26e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:29.795545 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-6bd1a505-ee82-403b-a69f-84794fab4bee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:29.805641 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:29.837542 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-73af1bd3-874e-42fc-b710-c189c0f26e43 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:29.840022 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-73af1bd3-874e-42fc-b710-c189c0f26e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:29.885237 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1a505-ee82-403b-a69f-84794fab4bee admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:29.891380 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba3169-45dd-42df-89a8-288a5ec740dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:29.892726 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba3169-45dd-42df-89a8-288a5ec740dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:29.893189 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba3169-45dd-42df-89a8-288a5ec740dd admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:29.893649 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba3169-45dd-42df-89a8-288a5ec740dd admin admin] PATH_INFO: `/v3/projects/aaae1e0f2ab8493cb12abbaf01f1833a` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:29.895301 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba3169-45dd-42df-89a8-288a5ec740dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaae1e0f2ab8493cb12abbaf01f1833a)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:29.898257 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba3169-45dd-42df-89a8-288a5ec740dd admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:29.944423 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1027/2061] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:49:29.961392 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af1bd3-874e-42fc-b710-c189c0f26e43 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:29.963036 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-73af1bd3-874e-42fc-b710-c189c0f26e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:29.972843 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-49ba3169-45dd-42df-89a8-288a5ec740dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaae1e0f2ab8493cb12abbaf01f1833a', 'request_id': 'req-49ba3169-45dd-42df-89a8-288a5ec740dd'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:29.976156 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:29.987055 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-49ba3169-45dd-42df-89a8-288a5ec740dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaae1e0f2ab8493cb12abbaf01f1833a', 'request_id': 'req-49ba3169-45dd-42df-89a8-288a5ec740dd'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:29.998171 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-49ba3169-45dd-42df-89a8-288a5ec740dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaae1e0f2ab8493cb12abbaf01f1833a', 'request_id': 'req-49ba3169-45dd-42df-89a8-288a5ec740dd'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:30.031400 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-49ba3169-45dd-42df-89a8-288a5ec740dd admin admin] The token cache is being invalidate because project aaae1e0f2ab8493cb12abbaf01f1833a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:49:30.031955 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-49ba3169-45dd-42df-89a8-288a5ec740dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:30.033114 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-49ba3169-45dd-42df-89a8-288a5ec740dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:30.035060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1035/2062] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:49:29 2022] DELETE /identity/v3/projects/aaae1e0f2ab8493cb12abbaf01f1833a => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:49:30.039988 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bf01a-3b58-4923-aad9-651b11932469 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:30.040319 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bf01a-3b58-4923-aad9-651b11932469 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:30.040643 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af1bd3-874e-42fc-b710-c189c0f26e43 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:30.040744 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bf01a-3b58-4923-aad9-651b11932469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:30.064681 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-0c0bf01a-3b58-4923-aad9-651b11932469 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:30.066019 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-0c0bf01a-3b58-4923-aad9-651b11932469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:30.099399 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63d7a9-06d0-4770-bfdc-46f6327bb1d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:30.100021 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63d7a9-06d0-4770-bfdc-46f6327bb1d8 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:30.100297 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63d7a9-06d0-4770-bfdc-46f6327bb1d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:30.100564 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63d7a9-06d0-4770-bfdc-46f6327bb1d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:30.102352 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-7a63d7a9-06d0-4770-bfdc-46f6327bb1d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:30.130609 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1036/2063] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:49:30.138665 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a43609-eaa1-4071-878d-31d0b951f99c None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:30.138951 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a43609-eaa1-4071-878d-31d0b951f99c None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:30.139220 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a43609-eaa1-4071-878d-31d0b951f99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:30.158994 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-e7a43609-eaa1-4071-878d-31d0b951f99c None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:30.160203 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-e7a43609-eaa1-4071-878d-31d0b951f99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:30.168099 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a63d7a9-06d0-4770-bfdc-46f6327bb1d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:30.168224 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1037/2064] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:49:30.169567 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a63d7a9-06d0-4770-bfdc-46f6327bb1d8 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:30.172326 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a43609-eaa1-4071-878d-31d0b951f99c None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:30.180298 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1028/2065] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:49:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:49:30.182184 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a43609-eaa1-4071-878d-31d0b951f99c None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:30.186850 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6236c0-2c2f-4ffc-b62b-1ece5e31d236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:30.187451 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6236c0-2c2f-4ffc-b62b-1ece5e31d236 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:30.187729 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6236c0-2c2f-4ffc-b62b-1ece5e31d236 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:30.188000 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6236c0-2c2f-4ffc-b62b-1ece5e31d236 admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:30.188711 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6236c0-2c2f-4ffc-b62b-1ece5e31d236 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:49:30.188973 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6236c0-2c2f-4ffc-b62b-1ece5e31d236 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:30.190601 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6236c0-2c2f-4ffc-b62b-1ece5e31d236 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:30.202337 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1038/2066] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:49:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:49:30.207054 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63d7a9-06d0-4770-bfdc-46f6327bb1d8 service neutron] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:30.217024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63d7a9-06d0-4770-bfdc-46f6327bb1d8 service neutron] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:30.223731 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf5c102-1322-4147-b903-38be66f5ae55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:30.224532 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5c102-1322-4147-b903-38be66f5ae55 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:30.224691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5c102-1322-4147-b903-38be66f5ae55 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:30.224963 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5c102-1322-4147-b903-38be66f5ae55 admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:30.225683 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf5c102-1322-4147-b903-38be66f5ae55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:30.227655 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf5c102-1322-4147-b903-38be66f5ae55 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:30.238277 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:30.257871 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1029/2067] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:49:30 2022] POST /identity/v3/projects => generated 393 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:49:30.262116 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6236c0-2c2f-4ffc-b62b-1ece5e31d236 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:30.273857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6236c0-2c2f-4ffc-b62b-1ece5e31d236 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:30.365599 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5b2d93-4a6c-467e-bc50-7ff533e7cede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:30.366657 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b2d93-4a6c-467e-bc50-7ff533e7cede admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:30.367043 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b2d93-4a6c-467e-bc50-7ff533e7cede admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:30.367423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b2d93-4a6c-467e-bc50-7ff533e7cede admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:30.368967 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5b2d93-4a6c-467e-bc50-7ff533e7cede admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:30.375145 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5b2d93-4a6c-467e-bc50-7ff533e7cede admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:30.429180 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1039/2068] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:49:30 2022] POST /identity/v3/users => generated 297 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:49:30.433607 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf5c102-1322-4147-b903-38be66f5ae55 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:30.446320 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf5c102-1322-4147-b903-38be66f5ae55 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:30.462447 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc924a2b-4a6d-474d-8f28-fbfa3c3d8f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:30.463181 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc924a2b-4a6d-474d-8f28-fbfa3c3d8f07 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:30.463514 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc924a2b-4a6d-474d-8f28-fbfa3c3d8f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:30.463785 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc924a2b-4a6d-474d-8f28-fbfa3c3d8f07 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:30.464548 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc924a2b-4a6d-474d-8f28-fbfa3c3d8f07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:30.466635 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc924a2b-4a6d-474d-8f28-fbfa3c3d8f07 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:30.481316 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1030/2069] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:49:30 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:49:30.484707 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5b2d93-4a6c-467e-bc50-7ff533e7cede admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:30.497739 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5b2d93-4a6c-467e-bc50-7ff533e7cede admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:30.504528 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f55b5-7c42-4c38-a65d-00592c078a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:30.505579 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f55b5-7c42-4c38-a65d-00592c078a7a admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:30.505579 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f55b5-7c42-4c38-a65d-00592c078a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:30.505873 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f55b5-7c42-4c38-a65d-00592c078a7a admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:30.506645 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f55b5-7c42-4c38-a65d-00592c078a7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:30.508154 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f55b5-7c42-4c38-a65d-00592c078a7a admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:30.527956 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1040/2070] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:49:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:49:30.529393 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc924a2b-4a6d-474d-8f28-fbfa3c3d8f07 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:30.545957 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc924a2b-4a6d-474d-8f28-fbfa3c3d8f07 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:30.554731 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5b530-a104-4658-854c-57f2ac3d7ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:30.555515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5b530-a104-4658-854c-57f2ac3d7ace admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:30.555801 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5b530-a104-4658-854c-57f2ac3d7ace admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:30.556100 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5b530-a104-4658-854c-57f2ac3d7ace admin admin] PATH_INFO: `/v3/projects/24b951d69396403b9f4b88cea1f3fa76/users/6367371496f04cc2a7c105fbbed68dca/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:30.566494 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f55b5-7c42-4c38-a65d-00592c078a7a admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:30.569145 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-635f55b5-7c42-4c38-a65d-00592c078a7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:30.586187 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f55b5-7c42-4c38-a65d-00592c078a7a admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:30.593177 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab4ea7-5966-4cc3-b263-af79b7909d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:30.594037 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab4ea7-5966-4cc3-b263-af79b7909d5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:30.594181 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab4ea7-5966-4cc3-b263-af79b7909d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:30.594430 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab4ea7-5966-4cc3-b263-af79b7909d5a admin admin] PATH_INFO: `/v3/projects/3c7b513237474dd9b7854741a9176e2c` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:30.595845 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eab4ea7-5966-4cc3-b263-af79b7909d5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c7b513237474dd9b7854741a9176e2c)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:30.601527 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a5b530-a104-4658-854c-57f2ac3d7ace admin admin] RBAC: Authorizing `identity:create_grant(project_id=24b951d69396403b9f4b88cea1f3fa76, user_id=6367371496f04cc2a7c105fbbed68dca, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:30.606435 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a5b530-a104-4658-854c-57f2ac3d7ace admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:30.614864 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eab4ea7-5966-4cc3-b263-af79b7909d5a admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:30.656168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1031/2071] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:49:30 2022] PUT /identity/v3/projects/24b951d69396403b9f4b88cea1f3fa76/users/6367371496f04cc2a7c105fbbed68dca/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:49:30.662577 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52019d-e812-4013-bc39-712d1b8b9397 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:30.663513 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52019d-e812-4013-bc39-712d1b8b9397 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:30.664024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52019d-e812-4013-bc39-712d1b8b9397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:30.687114 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:30.714723 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-3eab4ea7-5966-4cc3-b263-af79b7909d5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c7b513237474dd9b7854741a9176e2c', 'request_id': 'req-3eab4ea7-5966-4cc3-b263-af79b7909d5a'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:30.717714 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-5c52019d-e812-4013-bc39-712d1b8b9397 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:30.720598 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-5c52019d-e812-4013-bc39-712d1b8b9397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:30.723623 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-3eab4ea7-5966-4cc3-b263-af79b7909d5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c7b513237474dd9b7854741a9176e2c', 'request_id': 'req-3eab4ea7-5966-4cc3-b263-af79b7909d5a'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:30.751281 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-3eab4ea7-5966-4cc3-b263-af79b7909d5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c7b513237474dd9b7854741a9176e2c', 'request_id': 'req-3eab4ea7-5966-4cc3-b263-af79b7909d5a'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:30.775380 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-3eab4ea7-5966-4cc3-b263-af79b7909d5a admin admin] The token cache is being invalidate because project 3c7b513237474dd9b7854741a9176e2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:49:30.775624 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-3eab4ea7-5966-4cc3-b263-af79b7909d5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:30.776430 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-3eab4ea7-5966-4cc3-b263-af79b7909d5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:30.778763 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1041/2072] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:49:30 2022] DELETE /identity/v3/projects/3c7b513237474dd9b7854741a9176e2c => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:49:30.793767 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab4ea7-5966-4cc3-b263-af79b7909d5a admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:30.795724 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-3eab4ea7-5966-4cc3-b263-af79b7909d5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:30.827311 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1032/2073] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:49:30.839548 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52019d-e812-4013-bc39-712d1b8b9397 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:30.841432 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-5c52019d-e812-4013-bc39-712d1b8b9397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:30.901271 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab4ea7-5966-4cc3-b263-af79b7909d5a admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:30.909876 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d3ef8-866f-4e35-bfbe-c6e7c9ffc990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:30.910679 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d3ef8-866f-4e35-bfbe-c6e7c9ffc990 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:30.911082 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d3ef8-866f-4e35-bfbe-c6e7c9ffc990 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:30.911702 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d3ef8-866f-4e35-bfbe-c6e7c9ffc990 admin admin] PATH_INFO: `/v3/users/754e963325404349b364125c4e73b508` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:30.913040 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d3ef8-866f-4e35-bfbe-c6e7c9ffc990 admin admin] RBAC: Authorizing `identity:delete_user(user_id=754e963325404349b364125c4e73b508)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:30.915197 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d3ef8-866f-4e35-bfbe-c6e7c9ffc990 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:30.916912 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52019d-e812-4013-bc39-712d1b8b9397 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:30.921349 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa8069e-06e7-483e-83c7-3fb1040d8e14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:30.922084 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8069e-06e7-483e-83c7-3fb1040d8e14 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:30.922442 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8069e-06e7-483e-83c7-3fb1040d8e14 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:30.922936 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8069e-06e7-483e-83c7-3fb1040d8e14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:30.935064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa8069e-06e7-483e-83c7-3fb1040d8e14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:30.936441 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa8069e-06e7-483e-83c7-3fb1040d8e14 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:30.948994 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1033/2074] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:49:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:49:31.002244 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-d84d3ef8-866f-4e35-bfbe-c6e7c9ffc990 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '754e963325404349b364125c4e73b508'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:31.020180 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-d84d3ef8-866f-4e35-bfbe-c6e7c9ffc990 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '754e963325404349b364125c4e73b508'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:31.030024 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-d84d3ef8-866f-4e35-bfbe-c6e7c9ffc990 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '754e963325404349b364125c4e73b508'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:31.030855 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-d84d3ef8-866f-4e35-bfbe-c6e7c9ffc990 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '754e963325404349b364125c4e73b508'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:31.031688 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-d84d3ef8-866f-4e35-bfbe-c6e7c9ffc990 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '754e963325404349b364125c4e73b508'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:31.095580 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1042/2075] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:49:30 2022] DELETE /identity/v3/users/754e963325404349b364125c4e73b508 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:49:31.682643 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa8069e-06e7-483e-83c7-3fb1040d8e14 service neutron] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:31.702484 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa8069e-06e7-483e-83c7-3fb1040d8e14 service neutron] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:31.810774 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:31.821020 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd34cb6-22d1-4729-b606-24e6738c4a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:31.821902 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd34cb6-22d1-4729-b606-24e6738c4a31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:31.822746 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd34cb6-22d1-4729-b606-24e6738c4a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:31.823104 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd34cb6-22d1-4729-b606-24e6738c4a31 admin admin] PATH_INFO: `/v3/projects/b3e4320d05314707b85180f214009c18` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:31.824863 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd34cb6-22d1-4729-b606-24e6738c4a31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3e4320d05314707b85180f214009c18)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:31.829729 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd34cb6-22d1-4729-b606-24e6738c4a31 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:31.942389 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-1fd34cb6-22d1-4729-b606-24e6738c4a31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3e4320d05314707b85180f214009c18', 'request_id': 'req-1fd34cb6-22d1-4729-b606-24e6738c4a31'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:31.958602 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-1fd34cb6-22d1-4729-b606-24e6738c4a31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3e4320d05314707b85180f214009c18', 'request_id': 'req-1fd34cb6-22d1-4729-b606-24e6738c4a31'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:31.967257 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-1fd34cb6-22d1-4729-b606-24e6738c4a31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3e4320d05314707b85180f214009c18', 'request_id': 'req-1fd34cb6-22d1-4729-b606-24e6738c4a31'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:31.982594 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-1fd34cb6-22d1-4729-b606-24e6738c4a31 admin admin] The token cache is being invalidate because project b3e4320d05314707b85180f214009c18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:49:31.983646 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-1fd34cb6-22d1-4729-b606-24e6738c4a31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:31.985686 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-1fd34cb6-22d1-4729-b606-24e6738c4a31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:31.992340 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1034/2076] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:49:31 2022] DELETE /identity/v3/projects/b3e4320d05314707b85180f214009c18 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:49:31.994226 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f19e7d-30c7-441c-b1ac-01781e72acbd None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:31.994771 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f19e7d-30c7-441c-b1ac-01781e72acbd None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:31.994992 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f19e7d-30c7-441c-b1ac-01781e72acbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:32.020727 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-b0f19e7d-30c7-441c-b1ac-01781e72acbd None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:32.023175 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-b0f19e7d-30c7-441c-b1ac-01781e72acbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:32.089872 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:32.124093 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1043/2077] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:49:32.132778 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361aeac-9bd3-4562-afed-23d2027ee476 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:32.133060 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361aeac-9bd3-4562-afed-23d2027ee476 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:32.133364 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361aeac-9bd3-4562-afed-23d2027ee476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:32.154513 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-1361aeac-9bd3-4562-afed-23d2027ee476 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:32.155880 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-1361aeac-9bd3-4562-afed-23d2027ee476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:32.168736 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1035/2078] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:49:32.169774 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f19e7d-30c7-441c-b1ac-01781e72acbd None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:32.181674 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f19e7d-30c7-441c-b1ac-01781e72acbd None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:32.186468 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1fa3e8-f493-47d1-ae35-f61e76856a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:32.187085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fa3e8-f493-47d1-ae35-f61e76856a51 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:32.187361 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fa3e8-f493-47d1-ae35-f61e76856a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:32.187769 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fa3e8-f493-47d1-ae35-f61e76856a51 admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:32.189019 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1fa3e8-f493-47d1-ae35-f61e76856a51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:49:32.189346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1fa3e8-f493-47d1-ae35-f61e76856a51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:32.191052 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1fa3e8-f493-47d1-ae35-f61e76856a51 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:32.211694 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1044/2079] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:49:32.217067 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361aeac-9bd3-4562-afed-23d2027ee476 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:32.230503 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361aeac-9bd3-4562-afed-23d2027ee476 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:32.238398 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbddfb9-6a74-4962-87cb-fe649f78fb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:32.239349 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbddfb9-6a74-4962-87cb-fe649f78fb8e admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:32.239850 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbddfb9-6a74-4962-87cb-fe649f78fb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:32.240209 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbddfb9-6a74-4962-87cb-fe649f78fb8e admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:32.240923 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbddfb9-6a74-4962-87cb-fe649f78fb8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:32.243541 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbddfb9-6a74-4962-87cb-fe649f78fb8e admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:32.289996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449572d-1a59-4dc2-b546-bb09dc1df939 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:32.290780 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449572d-1a59-4dc2-b546-bb09dc1df939 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:32.291228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449572d-1a59-4dc2-b546-bb09dc1df939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:32.306383 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1036/2080] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:49:32 2022] POST /identity/v3/projects => generated 387 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:49:32.310408 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbddfb9-6a74-4962-87cb-fe649f78fb8e admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:32.312959 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-a449572d-1a59-4dc2-b546-bb09dc1df939 None None] MFA Rules not processed for user `0f645ba95eb64bb2b23d0d38a141307a`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:32.315036 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-a449572d-1a59-4dc2-b546-bb09dc1df939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:32.387695 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbddfb9-6a74-4962-87cb-fe649f78fb8e admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:32.395686 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1045/2081] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:49:32.411968 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b195c-85e9-49bc-86ee-401d4d32f259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:32.413003 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b195c-85e9-49bc-86ee-401d4d32f259 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:32.413480 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b195c-85e9-49bc-86ee-401d4d32f259 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:32.414071 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b195c-85e9-49bc-86ee-401d4d32f259 admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:32.414986 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92b195c-85e9-49bc-86ee-401d4d32f259 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:32.431827 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92b195c-85e9-49bc-86ee-401d4d32f259 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:32.497383 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1037/2082] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:49:32 2022] POST /identity/v3/users => generated 294 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:49:32.498914 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449572d-1a59-4dc2-b546-bb09dc1df939 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:32.517656 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449572d-1a59-4dc2-b546-bb09dc1df939 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:32.526021 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d4587-428e-4370-9ba2-bb641e3aa3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:32.527016 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d4587-428e-4370-9ba2-bb641e3aa3b5 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:32.527435 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d4587-428e-4370-9ba2-bb641e3aa3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:32.527888 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d4587-428e-4370-9ba2-bb641e3aa3b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:32.528931 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343d4587-428e-4370-9ba2-bb641e3aa3b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:32.532869 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343d4587-428e-4370-9ba2-bb641e3aa3b5 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:32.550170 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1046/2083] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:49:32 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:49:32.551446 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b195c-85e9-49bc-86ee-401d4d32f259 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:32.567256 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b195c-85e9-49bc-86ee-401d4d32f259 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:32.572547 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c55c82-e09f-4aab-99a7-22860efa3888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:32.573156 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c55c82-e09f-4aab-99a7-22860efa3888 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:32.573462 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c55c82-e09f-4aab-99a7-22860efa3888 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:32.573730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c55c82-e09f-4aab-99a7-22860efa3888 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:32.574372 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c55c82-e09f-4aab-99a7-22860efa3888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:32.576108 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c55c82-e09f-4aab-99a7-22860efa3888 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:32.602591 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1038/2084] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:49:32 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:49:32.606158 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d4587-428e-4370-9ba2-bb641e3aa3b5 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:32.620533 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d4587-428e-4370-9ba2-bb641e3aa3b5 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:32.627901 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6c0b9-6172-4b13-abb4-b139fa1e2c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:32.628631 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6c0b9-6172-4b13-abb4-b139fa1e2c41 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:32.628913 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6c0b9-6172-4b13-abb4-b139fa1e2c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:32.629184 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6c0b9-6172-4b13-abb4-b139fa1e2c41 admin admin] PATH_INFO: `/v3/projects/a12d68aa73de426bae9e85c528a8fb01/users/c77733ddfb8b44f59e33a79416298bfb/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:32.651144 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6c0b9-6172-4b13-abb4-b139fa1e2c41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a12d68aa73de426bae9e85c528a8fb01, user_id=c77733ddfb8b44f59e33a79416298bfb, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:32.654492 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6c0b9-6172-4b13-abb4-b139fa1e2c41 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:32.691343 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1047/2085] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:49:32 2022] PUT /identity/v3/projects/a12d68aa73de426bae9e85c528a8fb01/users/c77733ddfb8b44f59e33a79416298bfb/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:49:32.693132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa312d-9150-48a8-97a1-e4a972b22f31 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:32.693680 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa312d-9150-48a8-97a1-e4a972b22f31 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:32.694016 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa312d-9150-48a8-97a1-e4a972b22f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:32.733933 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-c0fa312d-9150-48a8-97a1-e4a972b22f31 None None] MFA Rules not processed for user `c77733ddfb8b44f59e33a79416298bfb`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:32.735268 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-c0fa312d-9150-48a8-97a1-e4a972b22f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:32.776679 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:32.850319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1039/2086] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:32 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:49:32.854861 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6c0b9-6172-4b13-abb4-b139fa1e2c41 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:32.948238 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6c0b9-6172-4b13-abb4-b139fa1e2c41 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:32.954116 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a97c84-176d-4f22-ad93-8fa3a6f0faf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:32.954744 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a97c84-176d-4f22-ad93-8fa3a6f0faf5 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:32.955023 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a97c84-176d-4f22-ad93-8fa3a6f0faf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:32.955294 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a97c84-176d-4f22-ad93-8fa3a6f0faf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:32.956011 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a97c84-176d-4f22-ad93-8fa3a6f0faf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:32.958282 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a97c84-176d-4f22-ad93-8fa3a6f0faf5 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:33.003775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1048/2087] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:49:32 2022] POST /identity/v3/projects => generated 385 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:49:33.007662 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa312d-9150-48a8-97a1-e4a972b22f31 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:33.021810 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa312d-9150-48a8-97a1-e4a972b22f31 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:33.159218 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161bb666-852b-4f9d-8891-fab57809e98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:33.159834 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161bb666-852b-4f9d-8891-fab57809e98c admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:33.160119 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161bb666-852b-4f9d-8891-fab57809e98c admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:33.160455 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161bb666-852b-4f9d-8891-fab57809e98c admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:33.161103 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161bb666-852b-4f9d-8891-fab57809e98c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:33.163908 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161bb666-852b-4f9d-8891-fab57809e98c admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:33.208725 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1040/2088] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:49:33 2022] POST /identity/v3/users => generated 299 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:49:33.212339 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a97c84-176d-4f22-ad93-8fa3a6f0faf5 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:33.224247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a97c84-176d-4f22-ad93-8fa3a6f0faf5 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:33.233980 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff739f2-21aa-49cb-a75e-1b5d069ab863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:33.234604 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff739f2-21aa-49cb-a75e-1b5d069ab863 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:33.236017 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff739f2-21aa-49cb-a75e-1b5d069ab863 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:33.236465 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff739f2-21aa-49cb-a75e-1b5d069ab863 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:33.237240 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff739f2-21aa-49cb-a75e-1b5d069ab863 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:33.239885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff739f2-21aa-49cb-a75e-1b5d069ab863 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:33.262938 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1049/2089] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 22:49:33.264341 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161bb666-852b-4f9d-8891-fab57809e98c admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:33.277787 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161bb666-852b-4f9d-8891-fab57809e98c admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:33.291722 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f097d-e7e4-409f-afb9-b3633ba8c43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:33.292794 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f097d-e7e4-409f-afb9-b3633ba8c43d admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:33.293272 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f097d-e7e4-409f-afb9-b3633ba8c43d admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:33.293753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f097d-e7e4-409f-afb9-b3633ba8c43d admin admin] PATH_INFO: `/v3/projects/2e55d9158ab14830bb5bdf55dc40f056/users/ce42fff839c749f7b780c56528bf123f/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:33.319214 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7f097d-e7e4-409f-afb9-b3633ba8c43d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e55d9158ab14830bb5bdf55dc40f056, user_id=ce42fff839c749f7b780c56528bf123f, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:33.328983 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7f097d-e7e4-409f-afb9-b3633ba8c43d admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:33.358392 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1041/2090] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:49:33 2022] PUT /identity/v3/projects/2e55d9158ab14830bb5bdf55dc40f056/users/ce42fff839c749f7b780c56528bf123f/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:49:33.363230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff739f2-21aa-49cb-a75e-1b5d069ab863 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:33.375544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff739f2-21aa-49cb-a75e-1b5d069ab863 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:33.390058 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:33.479351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc549b71-6219-4737-81f7-499b50cf9248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:33.479967 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc549b71-6219-4737-81f7-499b50cf9248 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:33.480247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc549b71-6219-4737-81f7-499b50cf9248 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:33.480536 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc549b71-6219-4737-81f7-499b50cf9248 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:33.481174 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc549b71-6219-4737-81f7-499b50cf9248 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:33.482817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc549b71-6219-4737-81f7-499b50cf9248 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:33.499576 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1050/2091] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 22:49:33.510656 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f097d-e7e4-409f-afb9-b3633ba8c43d admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:33.526882 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f097d-e7e4-409f-afb9-b3633ba8c43d admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:33.540193 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad5ea3-5225-4b0c-8675-724a7c12e219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:33.550603 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad5ea3-5225-4b0c-8675-724a7c12e219 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:33.550896 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad5ea3-5225-4b0c-8675-724a7c12e219 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:33.551186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad5ea3-5225-4b0c-8675-724a7c12e219 admin admin] PATH_INFO: `/v3/projects/2e55d9158ab14830bb5bdf55dc40f056/users/ce42fff839c749f7b780c56528bf123f/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:33.558216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad5ea3-5225-4b0c-8675-724a7c12e219 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e55d9158ab14830bb5bdf55dc40f056, user_id=ce42fff839c749f7b780c56528bf123f, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:33.569020 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad5ea3-5225-4b0c-8675-724a7c12e219 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:33.641949 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1042/2092] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:49:33 2022] PUT /identity/v3/projects/2e55d9158ab14830bb5bdf55dc40f056/users/ce42fff839c749f7b780c56528bf123f/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:49:33.646850 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa8ad9-9ffa-43f9-8620-84074e76c8cc None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:33.647645 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa8ad9-9ffa-43f9-8620-84074e76c8cc None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:33.648578 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa8ad9-9ffa-43f9-8620-84074e76c8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:33.720763 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-22aa8ad9-9ffa-43f9-8620-84074e76c8cc None None] MFA Rules not processed for user `ce42fff839c749f7b780c56528bf123f`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:33.722050 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-22aa8ad9-9ffa-43f9-8620-84074e76c8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:33.870435 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1051/2093] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:33 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:49:33.885202 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad5ea3-5225-4b0c-8675-724a7c12e219 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:33.891281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-13ad5ea3-5225-4b0c-8675-724a7c12e219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:33.976632 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad5ea3-5225-4b0c-8675-724a7c12e219 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:33.981465 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8598acb-9cf0-4bca-961b-9164373ed220 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:33.982321 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8598acb-9cf0-4bca-961b-9164373ed220 service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:33.982507 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8598acb-9cf0-4bca-961b-9164373ed220 service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:33.982781 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8598acb-9cf0-4bca-961b-9164373ed220 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:33.993399 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8598acb-9cf0-4bca-961b-9164373ed220 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:33.994923 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8598acb-9cf0-4bca-961b-9164373ed220 service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:34.007702 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1043/2094] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:49:33 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:49:34.184069 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aa8ad9-9ffa-43f9-8620-84074e76c8cc None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:34.268559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aa8ad9-9ffa-43f9-8620-84074e76c8cc None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:34.275463 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92773d-5f30-488b-b173-980d553aa11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:34.276235 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92773d-5f30-488b-b173-980d553aa11f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:34.276517 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92773d-5f30-488b-b173-980d553aa11f admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:34.276841 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92773d-5f30-488b-b173-980d553aa11f admin admin] PATH_INFO: `/v3/users/c77733ddfb8b44f59e33a79416298bfb` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:34.278802 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb92773d-5f30-488b-b173-980d553aa11f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c77733ddfb8b44f59e33a79416298bfb)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:34.281340 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb92773d-5f30-488b-b173-980d553aa11f admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:34.344176 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:34.361971 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-eb92773d-5f30-488b-b173-980d553aa11f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c77733ddfb8b44f59e33a79416298bfb'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:34.371425 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-eb92773d-5f30-488b-b173-980d553aa11f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c77733ddfb8b44f59e33a79416298bfb'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:34.380411 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-eb92773d-5f30-488b-b173-980d553aa11f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c77733ddfb8b44f59e33a79416298bfb'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:34.381149 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-eb92773d-5f30-488b-b173-980d553aa11f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c77733ddfb8b44f59e33a79416298bfb'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:34.381934 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-eb92773d-5f30-488b-b173-980d553aa11f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c77733ddfb8b44f59e33a79416298bfb'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:34.410204 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1052/2095] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:49:34 2022] DELETE /identity/v3/users/c77733ddfb8b44f59e33a79416298bfb => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:49:34.414737 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc93b3e-bd4b-41b4-98f4-ade32f5b1af3 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:34.415629 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc93b3e-bd4b-41b4-98f4-ade32f5b1af3 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:34.415972 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc93b3e-bd4b-41b4-98f4-ade32f5b1af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:34.449344 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-efc93b3e-bd4b-41b4-98f4-ade32f5b1af3 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:34.450718 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-efc93b3e-bd4b-41b4-98f4-ade32f5b1af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:34.472722 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:34.533326 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1044/2096] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:49:34.543526 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92773d-5f30-488b-b173-980d553aa11f admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:34.544948 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-eb92773d-5f30-488b-b173-980d553aa11f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:34.621977 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92773d-5f30-488b-b173-980d553aa11f admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:34.631971 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc47bc2-d094-42c6-8f14-79fd337df926 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:34.632717 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc47bc2-d094-42c6-8f14-79fd337df926 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:34.632887 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc47bc2-d094-42c6-8f14-79fd337df926 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:34.633163 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc47bc2-d094-42c6-8f14-79fd337df926 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:34.646180 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc47bc2-d094-42c6-8f14-79fd337df926 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:34.647718 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc47bc2-d094-42c6-8f14-79fd337df926 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:34.663050 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1053/2097] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:49:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:49:35.057140 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc93b3e-bd4b-41b4-98f4-ade32f5b1af3 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:35.060738 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-efc93b3e-bd4b-41b4-98f4-ade32f5b1af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:35.079784 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc93b3e-bd4b-41b4-98f4-ade32f5b1af3 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:35.093799 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bdc0d6-96da-445d-9712-64e9b3365283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:35.095046 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bdc0d6-96da-445d-9712-64e9b3365283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:35.095331 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bdc0d6-96da-445d-9712-64e9b3365283 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:35.095765 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bdc0d6-96da-445d-9712-64e9b3365283 admin admin] PATH_INFO: `/v3/projects/a12d68aa73de426bae9e85c528a8fb01` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:35.097783 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bdc0d6-96da-445d-9712-64e9b3365283 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a12d68aa73de426bae9e85c528a8fb01)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:35.099713 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bdc0d6-96da-445d-9712-64e9b3365283 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:35.164708 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-84bdc0d6-96da-445d-9712-64e9b3365283 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a12d68aa73de426bae9e85c528a8fb01', 'request_id': 'req-84bdc0d6-96da-445d-9712-64e9b3365283'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:35.186466 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-84bdc0d6-96da-445d-9712-64e9b3365283 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a12d68aa73de426bae9e85c528a8fb01', 'request_id': 'req-84bdc0d6-96da-445d-9712-64e9b3365283'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:35.194911 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-84bdc0d6-96da-445d-9712-64e9b3365283 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a12d68aa73de426bae9e85c528a8fb01', 'request_id': 'req-84bdc0d6-96da-445d-9712-64e9b3365283'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:35.209598 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-84bdc0d6-96da-445d-9712-64e9b3365283 admin admin] The token cache is being invalidate because project a12d68aa73de426bae9e85c528a8fb01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:49:35.209921 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-84bdc0d6-96da-445d-9712-64e9b3365283 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:35.210670 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-84bdc0d6-96da-445d-9712-64e9b3365283 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:35.212755 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1045/2098] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:49:35 2022] DELETE /identity/v3/projects/a12d68aa73de426bae9e85c528a8fb01 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:49:35.216518 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc47bc2-d094-42c6-8f14-79fd337df926 service neutron] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:35.229977 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc47bc2-d094-42c6-8f14-79fd337df926 service neutron] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:35.313743 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02834fae-4c39-4223-85ca-05c94accfbc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:35.314546 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02834fae-4c39-4223-85ca-05c94accfbc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:35.314906 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02834fae-4c39-4223-85ca-05c94accfbc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:35.315266 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02834fae-4c39-4223-85ca-05c94accfbc8 admin admin] PATH_INFO: `/v3/users/ce42fff839c749f7b780c56528bf123f` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:35.316638 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02834fae-4c39-4223-85ca-05c94accfbc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce42fff839c749f7b780c56528bf123f)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:35.318728 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02834fae-4c39-4223-85ca-05c94accfbc8 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:35.384065 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-02834fae-4c39-4223-85ca-05c94accfbc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce42fff839c749f7b780c56528bf123f'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:35.396866 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-02834fae-4c39-4223-85ca-05c94accfbc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce42fff839c749f7b780c56528bf123f'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:35.399349 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:35.407375 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-02834fae-4c39-4223-85ca-05c94accfbc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce42fff839c749f7b780c56528bf123f'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:35.408313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-02834fae-4c39-4223-85ca-05c94accfbc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce42fff839c749f7b780c56528bf123f'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:35.409311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-02834fae-4c39-4223-85ca-05c94accfbc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce42fff839c749f7b780c56528bf123f'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:35.431000 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1054/2099] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:49:35 2022] DELETE /identity/v3/users/ce42fff839c749f7b780c56528bf123f => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:49:35.956882 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bdc0d6-96da-445d-9712-64e9b3365283 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:35.958713 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-84bdc0d6-96da-445d-9712-64e9b3365283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:35.977514 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bdc0d6-96da-445d-9712-64e9b3365283 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:36.052973 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7087e2f5-bac9-47ab-a655-ba0a3b39322d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:36.053349 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087e2f5-bac9-47ab-a655-ba0a3b39322d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:36.053538 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087e2f5-bac9-47ab-a655-ba0a3b39322d admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:36.053816 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087e2f5-bac9-47ab-a655-ba0a3b39322d admin admin] PATH_INFO: `/v3/projects/2e55d9158ab14830bb5bdf55dc40f056` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:36.055326 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7087e2f5-bac9-47ab-a655-ba0a3b39322d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e55d9158ab14830bb5bdf55dc40f056)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:36.057411 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7087e2f5-bac9-47ab-a655-ba0a3b39322d admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:36.116543 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:36.117838 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-7087e2f5-bac9-47ab-a655-ba0a3b39322d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e55d9158ab14830bb5bdf55dc40f056', 'request_id': 'req-7087e2f5-bac9-47ab-a655-ba0a3b39322d'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:36.128588 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-7087e2f5-bac9-47ab-a655-ba0a3b39322d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e55d9158ab14830bb5bdf55dc40f056', 'request_id': 'req-7087e2f5-bac9-47ab-a655-ba0a3b39322d'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:36.134860 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-7087e2f5-bac9-47ab-a655-ba0a3b39322d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e55d9158ab14830bb5bdf55dc40f056', 'request_id': 'req-7087e2f5-bac9-47ab-a655-ba0a3b39322d'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:36.152258 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-7087e2f5-bac9-47ab-a655-ba0a3b39322d admin admin] The token cache is being invalidate because project 2e55d9158ab14830bb5bdf55dc40f056 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:49:36.152548 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-7087e2f5-bac9-47ab-a655-ba0a3b39322d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:36.153263 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-7087e2f5-bac9-47ab-a655-ba0a3b39322d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:36.155364 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1046/2100] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:49:35 2022] DELETE /identity/v3/projects/2e55d9158ab14830bb5bdf55dc40f056 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:49:36.160775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7da117-3e84-40db-97c2-f0f15e86bb0a None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:36.161147 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7da117-3e84-40db-97c2-f0f15e86bb0a None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:36.161471 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7da117-3e84-40db-97c2-f0f15e86bb0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:36.183600 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-7c7da117-3e84-40db-97c2-f0f15e86bb0a None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:36.184868 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-7c7da117-3e84-40db-97c2-f0f15e86bb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:36.259826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1055/2101] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:49:36.268747 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ebaf3-d4b1-44fb-803f-8316f7a36026 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:36.268747 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ebaf3-d4b1-44fb-803f-8316f7a36026 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:36.268998 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ebaf3-d4b1-44fb-803f-8316f7a36026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:36.302782 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-b64ebaf3-d4b1-44fb-803f-8316f7a36026 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:36.304129 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-b64ebaf3-d4b1-44fb-803f-8316f7a36026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:36.314328 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1047/2102] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:49:36.318784 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7da117-3e84-40db-97c2-f0f15e86bb0a None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:36.329389 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7da117-3e84-40db-97c2-f0f15e86bb0a None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:36.334224 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be93082-8972-4aae-a839-9042e397a370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:36.334926 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be93082-8972-4aae-a839-9042e397a370 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:36.335285 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be93082-8972-4aae-a839-9042e397a370 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:36.335640 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be93082-8972-4aae-a839-9042e397a370 admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:36.336394 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be93082-8972-4aae-a839-9042e397a370 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:49:36.336770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be93082-8972-4aae-a839-9042e397a370 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:36.338512 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be93082-8972-4aae-a839-9042e397a370 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:36.352676 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1056/2103] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:49:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:49:36.358494 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64ebaf3-d4b1-44fb-803f-8316f7a36026 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:36.368719 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64ebaf3-d4b1-44fb-803f-8316f7a36026 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:36.382431 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39168c9f-8102-44c4-8ec8-4d35ddf930a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:36.383075 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39168c9f-8102-44c4-8ec8-4d35ddf930a6 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:36.383355 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39168c9f-8102-44c4-8ec8-4d35ddf930a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:36.383628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39168c9f-8102-44c4-8ec8-4d35ddf930a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:36.384269 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39168c9f-8102-44c4-8ec8-4d35ddf930a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:36.386340 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39168c9f-8102-44c4-8ec8-4d35ddf930a6 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:36.430685 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be93082-8972-4aae-a839-9042e397a370 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:36.438401 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1048/2104] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:49:36 2022] POST /identity/v3/projects => generated 367 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:49:36.444719 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be93082-8972-4aae-a839-9042e397a370 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:36.526317 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48ed3b-fdfa-41a8-b30b-d8a074db7680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:36.527279 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48ed3b-fdfa-41a8-b30b-d8a074db7680 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:36.527633 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48ed3b-fdfa-41a8-b30b-d8a074db7680 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:36.528020 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48ed3b-fdfa-41a8-b30b-d8a074db7680 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:36.528780 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f48ed3b-fdfa-41a8-b30b-d8a074db7680 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:36.530947 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f48ed3b-fdfa-41a8-b30b-d8a074db7680 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:36.583988 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1057/2105] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:49:36 2022] POST /identity/v3/users => generated 284 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:49:36.585552 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39168c9f-8102-44c4-8ec8-4d35ddf930a6 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:36.597065 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39168c9f-8102-44c4-8ec8-4d35ddf930a6 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:36.602777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d6013-b504-45f1-a89b-ecdc4609e853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:36.603393 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d6013-b504-45f1-a89b-ecdc4609e853 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:36.603676 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d6013-b504-45f1-a89b-ecdc4609e853 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:36.603989 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d6013-b504-45f1-a89b-ecdc4609e853 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:36.604602 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d6013-b504-45f1-a89b-ecdc4609e853 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:36.606251 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d6013-b504-45f1-a89b-ecdc4609e853 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:36.621902 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1049/2106] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:49:36.622983 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48ed3b-fdfa-41a8-b30b-d8a074db7680 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:36.634354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48ed3b-fdfa-41a8-b30b-d8a074db7680 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:36.639797 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c51a8-2cfa-4e9d-b3de-1a2465fdcb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:36.640534 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c51a8-2cfa-4e9d-b3de-1a2465fdcb83 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:36.640952 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c51a8-2cfa-4e9d-b3de-1a2465fdcb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:36.641264 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c51a8-2cfa-4e9d-b3de-1a2465fdcb83 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:36.641950 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115c51a8-2cfa-4e9d-b3de-1a2465fdcb83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:36.644817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115c51a8-2cfa-4e9d-b3de-1a2465fdcb83 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:36.657346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1058/2107] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:49:36.661450 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d6013-b504-45f1-a89b-ecdc4609e853 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:36.671876 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d6013-b504-45f1-a89b-ecdc4609e853 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:36.678631 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dbf634-3809-4b8e-b839-cc0a1fa0fd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:36.679149 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbf634-3809-4b8e-b839-cc0a1fa0fd56 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:36.679431 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbf634-3809-4b8e-b839-cc0a1fa0fd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:36.679855 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbf634-3809-4b8e-b839-cc0a1fa0fd56 admin admin] PATH_INFO: `/v3/projects/564105aa66844d3ba6015f50cb44d5a6/users/391b4c5d3b784fe89e634bd7e88d13f0/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:36.704067 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dbf634-3809-4b8e-b839-cc0a1fa0fd56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=564105aa66844d3ba6015f50cb44d5a6, user_id=391b4c5d3b784fe89e634bd7e88d13f0, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:36.707006 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dbf634-3809-4b8e-b839-cc0a1fa0fd56 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:36.730621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1050/2108] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:49:36 2022] PUT /identity/v3/projects/564105aa66844d3ba6015f50cb44d5a6/users/391b4c5d3b784fe89e634bd7e88d13f0/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:49:36.742616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27dc04-9b3f-497d-9d62-9fb4ba0e6b0a None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:36.742907 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27dc04-9b3f-497d-9d62-9fb4ba0e6b0a None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:36.743228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27dc04-9b3f-497d-9d62-9fb4ba0e6b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:36.787204 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-2f27dc04-9b3f-497d-9d62-9fb4ba0e6b0a None None] MFA Rules not processed for user `391b4c5d3b784fe89e634bd7e88d13f0`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:36.789045 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-2f27dc04-9b3f-497d-9d62-9fb4ba0e6b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:36.831664 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:36.935608 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1059/2109] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:49:36.940687 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dbf634-3809-4b8e-b839-cc0a1fa0fd56 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:37.043919 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dbf634-3809-4b8e-b839-cc0a1fa0fd56 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:37.050125 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c892b26-0028-474b-9621-dc8cfa10e096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:37.051023 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c892b26-0028-474b-9621-dc8cfa10e096 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:37.051169 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c892b26-0028-474b-9621-dc8cfa10e096 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:37.051449 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c892b26-0028-474b-9621-dc8cfa10e096 admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:37.052159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c892b26-0028-474b-9621-dc8cfa10e096 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:37.054289 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c892b26-0028-474b-9621-dc8cfa10e096 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:37.087737 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1051/2110] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:49:36 2022] POST /identity/v3/projects => generated 367 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:49:37.091662 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f27dc04-9b3f-497d-9d62-9fb4ba0e6b0a None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:37.106176 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f27dc04-9b3f-497d-9d62-9fb4ba0e6b0a None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:37.178276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd148d8e-d5ea-4a4e-b24e-69a40655e9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:37.179219 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd148d8e-d5ea-4a4e-b24e-69a40655e9d3 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:37.179688 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd148d8e-d5ea-4a4e-b24e-69a40655e9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:37.180355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd148d8e-d5ea-4a4e-b24e-69a40655e9d3 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:37.181381 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd148d8e-d5ea-4a4e-b24e-69a40655e9d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:37.183679 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd148d8e-d5ea-4a4e-b24e-69a40655e9d3 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:37.224322 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1060/2111] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:49:37 2022] POST /identity/v3/users => generated 290 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:49:37.232834 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c892b26-0028-474b-9621-dc8cfa10e096 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:37.249005 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c892b26-0028-474b-9621-dc8cfa10e096 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:37.254236 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca759e3-6deb-40d4-9591-07e4d0df4cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:37.254978 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca759e3-6deb-40d4-9591-07e4d0df4cf5 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:37.255369 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca759e3-6deb-40d4-9591-07e4d0df4cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:37.255738 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca759e3-6deb-40d4-9591-07e4d0df4cf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:37.256465 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca759e3-6deb-40d4-9591-07e4d0df4cf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:37.258237 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca759e3-6deb-40d4-9591-07e4d0df4cf5 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:37.278293 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1052/2112] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:49:37.281737 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd148d8e-d5ea-4a4e-b24e-69a40655e9d3 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:37.292772 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd148d8e-d5ea-4a4e-b24e-69a40655e9d3 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:37.301177 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f05ab9-ad85-4424-9d0c-ab47f33df2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:37.301903 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f05ab9-ad85-4424-9d0c-ab47f33df2d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:37.302184 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f05ab9-ad85-4424-9d0c-ab47f33df2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:37.302459 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f05ab9-ad85-4424-9d0c-ab47f33df2d5 admin admin] PATH_INFO: `/v3/projects/1f2a65f0b0d746cf8e5479626f5fe8a8/users/740b4553cf414b158453208ae2fa4d86/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:37.326026 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f05ab9-ad85-4424-9d0c-ab47f33df2d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f2a65f0b0d746cf8e5479626f5fe8a8, user_id=740b4553cf414b158453208ae2fa4d86, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:37.329062 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f05ab9-ad85-4424-9d0c-ab47f33df2d5 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:37.343293 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1061/2113] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:49:37 2022] PUT /identity/v3/projects/1f2a65f0b0d746cf8e5479626f5fe8a8/users/740b4553cf414b158453208ae2fa4d86/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:49:37.347712 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca759e3-6deb-40d4-9591-07e4d0df4cf5 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:37.358695 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca759e3-6deb-40d4-9591-07e4d0df4cf5 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:37.452632 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:37.491596 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c9d76d-13b1-42cf-87ad-9c92d270e253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:37.494004 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9d76d-13b1-42cf-87ad-9c92d270e253 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:37.495661 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9d76d-13b1-42cf-87ad-9c92d270e253 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:37.496069 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9d76d-13b1-42cf-87ad-9c92d270e253 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:37.498242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c9d76d-13b1-42cf-87ad-9c92d270e253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:37.505457 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c9d76d-13b1-42cf-87ad-9c92d270e253 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:37.535417 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1053/2114] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:49:37.536189 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f05ab9-ad85-4424-9d0c-ab47f33df2d5 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:37.558300 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f05ab9-ad85-4424-9d0c-ab47f33df2d5 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:37.565326 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba706be1-2517-49b4-90f0-a5f880d2aa91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:37.566146 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba706be1-2517-49b4-90f0-a5f880d2aa91 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:37.566635 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba706be1-2517-49b4-90f0-a5f880d2aa91 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:37.594551 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba706be1-2517-49b4-90f0-a5f880d2aa91 admin admin] PATH_INFO: `/v3/projects/1f2a65f0b0d746cf8e5479626f5fe8a8/users/740b4553cf414b158453208ae2fa4d86/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:37.618314 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba706be1-2517-49b4-90f0-a5f880d2aa91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f2a65f0b0d746cf8e5479626f5fe8a8, user_id=740b4553cf414b158453208ae2fa4d86, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:37.618314 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba706be1-2517-49b4-90f0-a5f880d2aa91 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:37.675833 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1062/2115] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:49:37 2022] PUT /identity/v3/projects/1f2a65f0b0d746cf8e5479626f5fe8a8/users/740b4553cf414b158453208ae2fa4d86/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:49:37.680342 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff57cb4-fc0c-4667-b224-bcb63b2c1c98 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:37.680682 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff57cb4-fc0c-4667-b224-bcb63b2c1c98 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:37.680904 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff57cb4-fc0c-4667-b224-bcb63b2c1c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:37.737081 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-1ff57cb4-fc0c-4667-b224-bcb63b2c1c98 None None] MFA Rules not processed for user `740b4553cf414b158453208ae2fa4d86`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:37.738527 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-1ff57cb4-fc0c-4667-b224-bcb63b2c1c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:37.887772 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1054/2116] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:37 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:49:37.901808 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba706be1-2517-49b4-90f0-a5f880d2aa91 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:37.903741 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-ba706be1-2517-49b4-90f0-a5f880d2aa91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:38.031734 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba706be1-2517-49b4-90f0-a5f880d2aa91 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:38.036311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2687b-7978-46d1-9752-8b8f2b0bedf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:38.037202 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2687b-7978-46d1-9752-8b8f2b0bedf2 service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:38.037676 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2687b-7978-46d1-9752-8b8f2b0bedf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:38.037954 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2687b-7978-46d1-9752-8b8f2b0bedf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:38.050185 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb2687b-7978-46d1-9752-8b8f2b0bedf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:38.051648 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb2687b-7978-46d1-9752-8b8f2b0bedf2 service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:38.062916 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1063/2117] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:49:37 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:49:38.089506 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083ca7c-0118-4b99-937a-899093b6151d None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:38.090571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083ca7c-0118-4b99-937a-899093b6151d None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:38.091046 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083ca7c-0118-4b99-937a-899093b6151d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:38.124084 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-5083ca7c-0118-4b99-937a-899093b6151d None None] MFA Rules not processed for user `6367371496f04cc2a7c105fbbed68dca`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:38.127187 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-5083ca7c-0118-4b99-937a-899093b6151d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:38.172110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2687b-7978-46d1-9752-8b8f2b0bedf2 service cinder] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:38.188454 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2687b-7978-46d1-9752-8b8f2b0bedf2 service cinder] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:38.201800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855ef3d2-ca33-4c24-b436-09f3d821e496 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:38.202434 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ef3d2-ca33-4c24-b436-09f3d821e496 service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:38.202716 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ef3d2-ca33-4c24-b436-09f3d821e496 service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:38.202990 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ef3d2-ca33-4c24-b436-09f3d821e496 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:38.244233 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:38.277628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1055/2118] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:38 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:49:38.288732 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c42203-3d62-42a8-bf24-099c11396b82 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:38.289023 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c42203-3d62-42a8-bf24-099c11396b82 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:38.289363 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c42203-3d62-42a8-bf24-099c11396b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:38.306041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855ef3d2-ca33-4c24-b436-09f3d821e496 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:38.307745 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855ef3d2-ca33-4c24-b436-09f3d821e496 service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:38.315292 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-73c42203-3d62-42a8-bf24-099c11396b82 None None] MFA Rules not processed for user `6367371496f04cc2a7c105fbbed68dca`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:38.316608 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-73c42203-3d62-42a8-bf24-099c11396b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:38.322820 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1064/2119] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:49:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:49:38.345243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1056/2120] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:38 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:49:38.358232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855ef3d2-ca33-4c24-b436-09f3d821e496 service cinder] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:38.361522 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-855ef3d2-ca33-4c24-b436-09f3d821e496 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:38.535138 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855ef3d2-ca33-4c24-b436-09f3d821e496 service cinder] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:38.544530 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9ff16-66c0-4333-b079-e868d5683b2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:38.545135 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9ff16-66c0-4333-b079-e868d5683b2a service nova] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:38.545547 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9ff16-66c0-4333-b079-e868d5683b2a service nova] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:38.545927 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9ff16-66c0-4333-b079-e868d5683b2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:38.558958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9ff16-66c0-4333-b079-e868d5683b2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:38.560699 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9ff16-66c0-4333-b079-e868d5683b2a service nova] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:38.581752 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1065/2121] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:49:38 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:49:39.636741 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c18355-8d71-4d56-a561-399e939c30cf None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:39.637188 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c18355-8d71-4d56-a561-399e939c30cf None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:39.637305 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c18355-8d71-4d56-a561-399e939c30cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:39.681940 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-10c18355-8d71-4d56-a561-399e939c30cf None None] MFA Rules not processed for user `6367371496f04cc2a7c105fbbed68dca`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:39.684408 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-10c18355-8d71-4d56-a561-399e939c30cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:39.693815 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1057/2122] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:39 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:49:44.951252 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9ff16-66c0-4333-b079-e868d5683b2a service nova] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:45.022915 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9ff16-66c0-4333-b079-e868d5683b2a service nova] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:45.027795 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6597a402-12f8-4867-a271-fb109b1bcff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:45.028521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597a402-12f8-4867-a271-fb109b1bcff5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:45.029087 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597a402-12f8-4867-a271-fb109b1bcff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:45.029245 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597a402-12f8-4867-a271-fb109b1bcff5 admin admin] PATH_INFO: `/v3/users/391b4c5d3b784fe89e634bd7e88d13f0` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:45.030873 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6597a402-12f8-4867-a271-fb109b1bcff5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=391b4c5d3b784fe89e634bd7e88d13f0)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:45.033077 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6597a402-12f8-4867-a271-fb109b1bcff5 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:45.085783 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-6597a402-12f8-4867-a271-fb109b1bcff5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '391b4c5d3b784fe89e634bd7e88d13f0'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:45.088262 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:45.093351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-6597a402-12f8-4867-a271-fb109b1bcff5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '391b4c5d3b784fe89e634bd7e88d13f0'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:45.101949 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-6597a402-12f8-4867-a271-fb109b1bcff5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '391b4c5d3b784fe89e634bd7e88d13f0'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:45.102599 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-6597a402-12f8-4867-a271-fb109b1bcff5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '391b4c5d3b784fe89e634bd7e88d13f0'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:45.103213 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-6597a402-12f8-4867-a271-fb109b1bcff5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '391b4c5d3b784fe89e634bd7e88d13f0'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:45.122260 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1066/2123] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:49:44 2022] DELETE /identity/v3/users/391b4c5d3b784fe89e634bd7e88d13f0 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:49:45.127581 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ff1bb-1f36-4e5e-ab19-8aebe8153d9e None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:45.128051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ff1bb-1f36-4e5e-ab19-8aebe8153d9e None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:45.128143 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ff1bb-1f36-4e5e-ab19-8aebe8153d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:45.146703 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-d09ff1bb-1f36-4e5e-ab19-8aebe8153d9e None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:45.147953 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-d09ff1bb-1f36-4e5e-ab19-8aebe8153d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:45.188342 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:45.207724 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1058/2124] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:49:45.218185 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6597a402-12f8-4867-a271-fb109b1bcff5 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:45.220318 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-6597a402-12f8-4867-a271-fb109b1bcff5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:45.279057 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6597a402-12f8-4867-a271-fb109b1bcff5 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:45.283475 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5dabe7-b24c-4325-a54e-b1f6e5ddd5aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:45.284090 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5dabe7-b24c-4325-a54e-b1f6e5ddd5aa service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:45.284372 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5dabe7-b24c-4325-a54e-b1f6e5ddd5aa service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:45.284640 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5dabe7-b24c-4325-a54e-b1f6e5ddd5aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:45.294241 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5dabe7-b24c-4325-a54e-b1f6e5ddd5aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:45.295673 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5dabe7-b24c-4325-a54e-b1f6e5ddd5aa service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:45.306544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1067/2125] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:49:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:49:45.940686 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ff1bb-1f36-4e5e-ab19-8aebe8153d9e None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:45.942262 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-d09ff1bb-1f36-4e5e-ab19-8aebe8153d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:45.956117 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ff1bb-1f36-4e5e-ab19-8aebe8153d9e None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:45.962232 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59650380-a179-42a4-b6e3-0a5b0157b1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:45.963172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59650380-a179-42a4-b6e3-0a5b0157b1ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:45.963663 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59650380-a179-42a4-b6e3-0a5b0157b1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:45.964145 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59650380-a179-42a4-b6e3-0a5b0157b1ae admin admin] PATH_INFO: `/v3/projects/564105aa66844d3ba6015f50cb44d5a6` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:45.965927 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59650380-a179-42a4-b6e3-0a5b0157b1ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=564105aa66844d3ba6015f50cb44d5a6)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:45.968084 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59650380-a179-42a4-b6e3-0a5b0157b1ae admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:46.021764 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-59650380-a179-42a4-b6e3-0a5b0157b1ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '564105aa66844d3ba6015f50cb44d5a6', 'request_id': 'req-59650380-a179-42a4-b6e3-0a5b0157b1ae'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:46.030013 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-59650380-a179-42a4-b6e3-0a5b0157b1ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '564105aa66844d3ba6015f50cb44d5a6', 'request_id': 'req-59650380-a179-42a4-b6e3-0a5b0157b1ae'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:46.040376 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-59650380-a179-42a4-b6e3-0a5b0157b1ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '564105aa66844d3ba6015f50cb44d5a6', 'request_id': 'req-59650380-a179-42a4-b6e3-0a5b0157b1ae'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:46.052834 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-59650380-a179-42a4-b6e3-0a5b0157b1ae admin admin] The token cache is being invalidate because project 564105aa66844d3ba6015f50cb44d5a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:49:46.053125 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-59650380-a179-42a4-b6e3-0a5b0157b1ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:46.053822 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-59650380-a179-42a4-b6e3-0a5b0157b1ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:46.055807 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1059/2126] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:49:45 2022] DELETE /identity/v3/projects/564105aa66844d3ba6015f50cb44d5a6 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:49:46.059768 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5dabe7-b24c-4325-a54e-b1f6e5ddd5aa service neutron] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:46.069923 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5dabe7-b24c-4325-a54e-b1f6e5ddd5aa service neutron] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:46.135293 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9dac3a-26c7-4ff2-b90c-9f2a182fd609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:46.135959 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9dac3a-26c7-4ff2-b90c-9f2a182fd609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:46.136246 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9dac3a-26c7-4ff2-b90c-9f2a182fd609 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:46.136523 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9dac3a-26c7-4ff2-b90c-9f2a182fd609 admin admin] PATH_INFO: `/v3/users/740b4553cf414b158453208ae2fa4d86` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:46.137722 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9dac3a-26c7-4ff2-b90c-9f2a182fd609 admin admin] RBAC: Authorizing `identity:delete_user(user_id=740b4553cf414b158453208ae2fa4d86)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:46.139736 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9dac3a-26c7-4ff2-b90c-9f2a182fd609 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:46.195313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-9d9dac3a-26c7-4ff2-b90c-9f2a182fd609 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '740b4553cf414b158453208ae2fa4d86'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:46.208607 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-9d9dac3a-26c7-4ff2-b90c-9f2a182fd609 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '740b4553cf414b158453208ae2fa4d86'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:46.216716 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-9d9dac3a-26c7-4ff2-b90c-9f2a182fd609 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '740b4553cf414b158453208ae2fa4d86'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:46.217437 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-9d9dac3a-26c7-4ff2-b90c-9f2a182fd609 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '740b4553cf414b158453208ae2fa4d86'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:46.218042 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-9d9dac3a-26c7-4ff2-b90c-9f2a182fd609 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '740b4553cf414b158453208ae2fa4d86'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:46.228387 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:46.241041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1068/2127] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:49:46 2022] DELETE /identity/v3/users/740b4553cf414b158453208ae2fa4d86 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:49:46.519625 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59650380-a179-42a4-b6e3-0a5b0157b1ae admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:46.521228 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-59650380-a179-42a4-b6e3-0a5b0157b1ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:46.535458 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59650380-a179-42a4-b6e3-0a5b0157b1ae admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:46.594175 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db03240f-5e8e-4e6c-a39d-b90be0ffa2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:46.594705 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03240f-5e8e-4e6c-a39d-b90be0ffa2da admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:46.594988 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03240f-5e8e-4e6c-a39d-b90be0ffa2da admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:46.595262 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03240f-5e8e-4e6c-a39d-b90be0ffa2da admin admin] PATH_INFO: `/v3/projects/1f2a65f0b0d746cf8e5479626f5fe8a8` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:46.596412 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db03240f-5e8e-4e6c-a39d-b90be0ffa2da admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f2a65f0b0d746cf8e5479626f5fe8a8)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:46.598385 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db03240f-5e8e-4e6c-a39d-b90be0ffa2da admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:46.649695 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-db03240f-5e8e-4e6c-a39d-b90be0ffa2da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f2a65f0b0d746cf8e5479626f5fe8a8', 'request_id': 'req-db03240f-5e8e-4e6c-a39d-b90be0ffa2da'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:46.657192 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-db03240f-5e8e-4e6c-a39d-b90be0ffa2da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f2a65f0b0d746cf8e5479626f5fe8a8', 'request_id': 'req-db03240f-5e8e-4e6c-a39d-b90be0ffa2da'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:46.662878 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-db03240f-5e8e-4e6c-a39d-b90be0ffa2da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f2a65f0b0d746cf8e5479626f5fe8a8', 'request_id': 'req-db03240f-5e8e-4e6c-a39d-b90be0ffa2da'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:46.669837 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:46.675609 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-db03240f-5e8e-4e6c-a39d-b90be0ffa2da admin admin] The token cache is being invalidate because project 1f2a65f0b0d746cf8e5479626f5fe8a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:49:46.675938 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-db03240f-5e8e-4e6c-a39d-b90be0ffa2da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:46.676839 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-db03240f-5e8e-4e6c-a39d-b90be0ffa2da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:49:46.679062 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1060/2128] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:49:46 2022] DELETE /identity/v3/projects/1f2a65f0b0d746cf8e5479626f5fe8a8 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:49:46.684488 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573a042-c345-4a49-8d4e-3fe4840dc89c None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:46.684773 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573a042-c345-4a49-8d4e-3fe4840dc89c None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:46.685047 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573a042-c345-4a49-8d4e-3fe4840dc89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:46.705563 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-f573a042-c345-4a49-8d4e-3fe4840dc89c None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:46.706765 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-f573a042-c345-4a49-8d4e-3fe4840dc89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:46.769693 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1069/2129] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:49:46.778358 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829e7e0-cceb-4f4f-a7f6-611b293da9bd None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:46.778665 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829e7e0-cceb-4f4f-a7f6-611b293da9bd None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:46.778948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829e7e0-cceb-4f4f-a7f6-611b293da9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:46.799767 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-7829e7e0-cceb-4f4f-a7f6-611b293da9bd None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:46.801539 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-7829e7e0-cceb-4f4f-a7f6-611b293da9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:46.809525 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1061/2130] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:49:46.814481 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f573a042-c345-4a49-8d4e-3fe4840dc89c None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:46.824519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f573a042-c345-4a49-8d4e-3fe4840dc89c None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:46.830311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791845c1-828b-4eab-8103-7fe5218ae0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:46.830932 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791845c1-828b-4eab-8103-7fe5218ae0d4 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:46.831217 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791845c1-828b-4eab-8103-7fe5218ae0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:46.831496 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791845c1-828b-4eab-8103-7fe5218ae0d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:46.832150 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791845c1-828b-4eab-8103-7fe5218ae0d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:49:46.832455 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791845c1-828b-4eab-8103-7fe5218ae0d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:46.834590 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791845c1-828b-4eab-8103-7fe5218ae0d4 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:46.846664 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1070/2131] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:49:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:49:46.851049 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7829e7e0-cceb-4f4f-a7f6-611b293da9bd None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:46.875212 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7829e7e0-cceb-4f4f-a7f6-611b293da9bd None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:46.880268 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da3c21-7b3d-4e27-a466-7686fa02bd84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:46.880876 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da3c21-7b3d-4e27-a466-7686fa02bd84 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:46.881153 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da3c21-7b3d-4e27-a466-7686fa02bd84 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:46.881456 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da3c21-7b3d-4e27-a466-7686fa02bd84 admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:46.882073 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da3c21-7b3d-4e27-a466-7686fa02bd84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:46.883999 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da3c21-7b3d-4e27-a466-7686fa02bd84 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:46.916566 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1062/2132] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:49:46 2022] POST /identity/v3/projects => generated 371 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:49:46.920669 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791845c1-828b-4eab-8103-7fe5218ae0d4 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:46.931779 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791845c1-828b-4eab-8103-7fe5218ae0d4 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:46.990511 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739fb547-517b-4678-8db5-fd9338c22377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:46.991126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fb547-517b-4678-8db5-fd9338c22377 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:46.991408 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fb547-517b-4678-8db5-fd9338c22377 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:46.991679 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fb547-517b-4678-8db5-fd9338c22377 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:46.992314 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739fb547-517b-4678-8db5-fd9338c22377 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:46.994331 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739fb547-517b-4678-8db5-fd9338c22377 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:47.021995 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1071/2133] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:49:46 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:49:47.025999 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da3c21-7b3d-4e27-a466-7686fa02bd84 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:47.036528 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da3c21-7b3d-4e27-a466-7686fa02bd84 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:47.041329 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f27a4-470d-47f8-babc-d7ae1d1096cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:47.041941 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f27a4-470d-47f8-babc-d7ae1d1096cb admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:47.042214 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f27a4-470d-47f8-babc-d7ae1d1096cb admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:47.042481 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f27a4-470d-47f8-babc-d7ae1d1096cb admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:47.043116 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f27a4-470d-47f8-babc-d7ae1d1096cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:47.044701 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f27a4-470d-47f8-babc-d7ae1d1096cb admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:47.056434 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1063/2134] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:49:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:49:47.060588 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739fb547-517b-4678-8db5-fd9338c22377 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:47.073255 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739fb547-517b-4678-8db5-fd9338c22377 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:47.078638 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6f310-79c5-4929-bdb4-6adbbe527355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:47.079249 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6f310-79c5-4929-bdb4-6adbbe527355 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:47.079533 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6f310-79c5-4929-bdb4-6adbbe527355 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:47.079808 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6f310-79c5-4929-bdb4-6adbbe527355 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:47.080450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee6f310-79c5-4929-bdb4-6adbbe527355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:47.082126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee6f310-79c5-4929-bdb4-6adbbe527355 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:47.092882 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1072/2135] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:49:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:49:47.097206 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f27a4-470d-47f8-babc-d7ae1d1096cb admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:47.107325 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f27a4-470d-47f8-babc-d7ae1d1096cb admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:47.112670 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb2f81f-49d3-49b8-9716-86e6cec3387a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:47.113505 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2f81f-49d3-49b8-9716-86e6cec3387a admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:47.113804 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2f81f-49d3-49b8-9716-86e6cec3387a admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:47.114078 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2f81f-49d3-49b8-9716-86e6cec3387a admin admin] PATH_INFO: `/v3/projects/2955b27b9b5748b3b51d4a7019bb7f81/users/08701e6ef90949a3a478307944db381a/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:47.129801 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb2f81f-49d3-49b8-9716-86e6cec3387a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2955b27b9b5748b3b51d4a7019bb7f81, user_id=08701e6ef90949a3a478307944db381a, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:47.132476 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb2f81f-49d3-49b8-9716-86e6cec3387a admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:47.145166 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1064/2136] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:49:47 2022] PUT /identity/v3/projects/2955b27b9b5748b3b51d4a7019bb7f81/users/08701e6ef90949a3a478307944db381a/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:49:47.150445 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a72a5a-5eec-4653-af92-4344d168c1da None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:47.150735 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a72a5a-5eec-4653-af92-4344d168c1da None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:47.151015 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a72a5a-5eec-4653-af92-4344d168c1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:47.170120 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-c3a72a5a-5eec-4653-af92-4344d168c1da None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:47.171341 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-c3a72a5a-5eec-4653-af92-4344d168c1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:47.224860 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:47.239065 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1073/2137] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:49:47.248914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb2f81f-49d3-49b8-9716-86e6cec3387a admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:47.251332 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-beb2f81f-49d3-49b8-9716-86e6cec3387a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:47.309046 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb2f81f-49d3-49b8-9716-86e6cec3387a admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:47.313514 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e1286-7d82-442c-b5f1-8464a68d2283 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:47.314144 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e1286-7d82-442c-b5f1-8464a68d2283 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:47.314422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e1286-7d82-442c-b5f1-8464a68d2283 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:47.314690 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e1286-7d82-442c-b5f1-8464a68d2283 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:47.324051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4e1286-7d82-442c-b5f1-8464a68d2283 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:47.325522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4e1286-7d82-442c-b5f1-8464a68d2283 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:47.335549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1065/2138] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:49:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:49:48.044258 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c738f7c-943e-4b58-807f-05a211f75744 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:48.044599 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c738f7c-943e-4b58-807f-05a211f75744 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:48.044888 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c738f7c-943e-4b58-807f-05a211f75744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:48.082387 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-1c738f7c-943e-4b58-807f-05a211f75744 None None] MFA Rules not processed for user `08701e6ef90949a3a478307944db381a`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:48.084894 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-1c738f7c-943e-4b58-807f-05a211f75744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:48.186947 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1074/2139] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:49:48.188470 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e1286-7d82-442c-b5f1-8464a68d2283 service neutron] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:48.199378 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e1286-7d82-442c-b5f1-8464a68d2283 service neutron] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:48.204689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07187670-f5e3-4a67-9830-4955eb80497b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:48.205418 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07187670-f5e3-4a67-9830-4955eb80497b admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:48.205752 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07187670-f5e3-4a67-9830-4955eb80497b admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:48.206101 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07187670-f5e3-4a67-9830-4955eb80497b admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:48.206642 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07187670-f5e3-4a67-9830-4955eb80497b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:48.208561 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07187670-f5e3-4a67-9830-4955eb80497b admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:48.250857 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1066/2140] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:49:48 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:49:48.252196 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c738f7c-943e-4b58-807f-05a211f75744 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:48.267954 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c738f7c-943e-4b58-807f-05a211f75744 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:48.324679 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4dd2e9-3896-4864-9f44-ff1c09cff762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:48.325320 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4dd2e9-3896-4864-9f44-ff1c09cff762 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:48.325609 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4dd2e9-3896-4864-9f44-ff1c09cff762 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:48.325883 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4dd2e9-3896-4864-9f44-ff1c09cff762 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:48.326507 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4dd2e9-3896-4864-9f44-ff1c09cff762 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:48.329432 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4dd2e9-3896-4864-9f44-ff1c09cff762 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:48.349502 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1075/2141] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:49:48 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:49:48.353726 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07187670-f5e3-4a67-9830-4955eb80497b admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:48.364350 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07187670-f5e3-4a67-9830-4955eb80497b admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:48.369061 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab115b-d767-4357-b33b-ca0716aa26d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:48.369698 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab115b-d767-4357-b33b-ca0716aa26d3 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:48.369983 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab115b-d767-4357-b33b-ca0716aa26d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:48.370256 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab115b-d767-4357-b33b-ca0716aa26d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:48.370894 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab115b-d767-4357-b33b-ca0716aa26d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:48.372495 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab115b-d767-4357-b33b-ca0716aa26d3 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:48.385118 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1067/2142] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:49:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:49:48.389345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4dd2e9-3896-4864-9f44-ff1c09cff762 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:48.400377 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4dd2e9-3896-4864-9f44-ff1c09cff762 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:48.404972 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6186cc0-1c57-408b-9490-092a4d98b29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:48.405643 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6186cc0-1c57-408b-9490-092a4d98b29d admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:48.406109 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6186cc0-1c57-408b-9490-092a4d98b29d admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:48.406705 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6186cc0-1c57-408b-9490-092a4d98b29d admin admin] PATH_INFO: `/v3/projects/5e536c3561a64d45be2a352cd5ee7b9a/users/042bd1e4ec78487390f04fed12b0c1c1/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:48.421841 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6186cc0-1c57-408b-9490-092a4d98b29d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e536c3561a64d45be2a352cd5ee7b9a, user_id=042bd1e4ec78487390f04fed12b0c1c1, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:48.425081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6186cc0-1c57-408b-9490-092a4d98b29d admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:48.442843 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1076/2143] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:49:48 2022] PUT /identity/v3/projects/5e536c3561a64d45be2a352cd5ee7b9a/users/042bd1e4ec78487390f04fed12b0c1c1/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:49:48.444075 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab115b-d767-4357-b33b-ca0716aa26d3 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:48.454784 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab115b-d767-4357-b33b-ca0716aa26d3 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:48.510843 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe165ab-9ab1-460f-992f-82daf0f5800c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:48.511453 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe165ab-9ab1-460f-992f-82daf0f5800c admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:48.511734 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe165ab-9ab1-460f-992f-82daf0f5800c admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:48.512007 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe165ab-9ab1-460f-992f-82daf0f5800c admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:48.512644 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe165ab-9ab1-460f-992f-82daf0f5800c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:48.514278 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe165ab-9ab1-460f-992f-82daf0f5800c admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:48.520115 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:48.526736 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1068/2144] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:49:48 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:49:48.530930 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6186cc0-1c57-408b-9490-092a4d98b29d admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:48.542032 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6186cc0-1c57-408b-9490-092a4d98b29d admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:48.547266 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01728d0-b671-4001-8c37-fc13e4c8755e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:48.547926 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01728d0-b671-4001-8c37-fc13e4c8755e admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:48.548208 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01728d0-b671-4001-8c37-fc13e4c8755e admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:48.548738 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01728d0-b671-4001-8c37-fc13e4c8755e admin admin] PATH_INFO: `/v3/projects/5e536c3561a64d45be2a352cd5ee7b9a/users/042bd1e4ec78487390f04fed12b0c1c1/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:48.550721 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01728d0-b671-4001-8c37-fc13e4c8755e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e536c3561a64d45be2a352cd5ee7b9a, user_id=042bd1e4ec78487390f04fed12b0c1c1, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:48.553401 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01728d0-b671-4001-8c37-fc13e4c8755e admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:48.564408 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1077/2145] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:49:48 2022] PUT /identity/v3/projects/5e536c3561a64d45be2a352cd5ee7b9a/users/042bd1e4ec78487390f04fed12b0c1c1/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:49:49.249755 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0c2ba-fdf9-4be4-b379-89f769cd94c9 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:49.250308 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0c2ba-fdf9-4be4-b379-89f769cd94c9 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:49.250478 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0c2ba-fdf9-4be4-b379-89f769cd94c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:49.310328 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-71d0c2ba-fdf9-4be4-b379-89f769cd94c9 None None] MFA Rules not processed for user `042bd1e4ec78487390f04fed12b0c1c1`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:49:49.311571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-71d0c2ba-fdf9-4be4-b379-89f769cd94c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:49.433333 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1069/2146] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:49:49 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:49:49.444168 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01728d0-b671-4001-8c37-fc13e4c8755e admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:49.446055 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-d01728d0-b671-4001-8c37-fc13e4c8755e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:49.502011 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:49:49.527053 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01728d0-b671-4001-8c37-fc13e4c8755e admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:49.531405 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e2ce6c-dfc9-4ca6-ae8a-1429d88d11d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:49.532152 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2ce6c-dfc9-4ca6-ae8a-1429d88d11d2 service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:49.532540 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2ce6c-dfc9-4ca6-ae8a-1429d88d11d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:49.532940 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2ce6c-dfc9-4ca6-ae8a-1429d88d11d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:49.611635 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e2ce6c-dfc9-4ca6-ae8a-1429d88d11d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:49.613371 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e2ce6c-dfc9-4ca6-ae8a-1429d88d11d2 service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:49.625232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1078/2147] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:49:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:49:52.141589 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d0c2ba-fdf9-4be4-b379-89f769cd94c9 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:49:52.142996 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-71d0c2ba-fdf9-4be4-b379-89f769cd94c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:49:52.154687 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d0c2ba-fdf9-4be4-b379-89f769cd94c9 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:49:52.159086 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49feba6d-139f-448d-bd03-708cdeeba0d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:49:52.159681 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49feba6d-139f-448d-bd03-708cdeeba0d0 service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:49:52.159961 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49feba6d-139f-448d-bd03-708cdeeba0d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:49:52.160229 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49feba6d-139f-448d-bd03-708cdeeba0d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:49:52.169330 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49feba6d-139f-448d-bd03-708cdeeba0d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:49:52.170754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49feba6d-139f-448d-bd03-708cdeeba0d0 service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:49:52.180647 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1070/2148] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:49:52 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 22:49:59.619532 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=31 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:50:02.176488 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=40 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:50:04.222656 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363b28f-11f0-405a-bd62-21a63841c712 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:04.223113 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363b28f-11f0-405a-bd62-21a63841c712 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:04.223217 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363b28f-11f0-405a-bd62-21a63841c712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:04.243274 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-b363b28f-11f0-405a-bd62-21a63841c712 None None] MFA Rules not processed for user `aff5e804c64345b298c745287a1d0a32`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:50:04.244467 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-b363b28f-11f0-405a-bd62-21a63841c712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:04.302493 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1079/2149] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 10 22:50:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:50:04.351023 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49feba6d-139f-448d-bd03-708cdeeba0d0 service cinder] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:04.352522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-49feba6d-139f-448d-bd03-708cdeeba0d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:04.432124 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49feba6d-139f-448d-bd03-708cdeeba0d0 service cinder] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:04.437261 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12885435-b578-4134-b9c7-7f38479948cc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e3a76df95a34b9fb9816f22b85d1071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:04.437897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12885435-b578-4134-b9c7-7f38479948cc service swift] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:04.438293 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12885435-b578-4134-b9c7-7f38479948cc service swift] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:04.438602 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12885435-b578-4134-b9c7-7f38479948cc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:04.446599 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12885435-b578-4134-b9c7-7f38479948cc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:04.448204 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12885435-b578-4134-b9c7-7f38479948cc service swift] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:04.456174 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1071/2150] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 10 22:50:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 22:50:05.077731 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d085449-6136-45a1-9f5c-86b8a75ed0f7 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:05.078104 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d085449-6136-45a1-9f5c-86b8a75ed0f7 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:05.078437 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d085449-6136-45a1-9f5c-86b8a75ed0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:05.104566 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-7d085449-6136-45a1-9f5c-86b8a75ed0f7 None None] MFA Rules not processed for user `aff5e804c64345b298c745287a1d0a32`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:50:05.106186 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-7d085449-6136-45a1-9f5c-86b8a75ed0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:05.115157 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1080/2151] 10.222.0.76 () {58 vars in 967 bytes} [Wed Aug 10 22:50:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:50:05.130983 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12885435-b578-4134-b9c7-7f38479948cc service swift] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:05.142506 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12885435-b578-4134-b9c7-7f38479948cc service swift] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:05.148064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69774f0-2a2f-4ee8-8de4-5aeb4218085c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e3a76df95a34b9fb9816f22b85d1071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:05.148560 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69774f0-2a2f-4ee8-8de4-5aeb4218085c service swift] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:05.148897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69774f0-2a2f-4ee8-8de4-5aeb4218085c service swift] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:05.152292 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69774f0-2a2f-4ee8-8de4-5aeb4218085c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:05.159502 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69774f0-2a2f-4ee8-8de4-5aeb4218085c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:05.162825 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69774f0-2a2f-4ee8-8de4-5aeb4218085c service swift] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:05.174876 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1072/2152] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 10 22:50:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 22:50:05.504135 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5376b-d32f-4e2f-991e-86596ed3e55b None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:05.505687 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5376b-d32f-4e2f-991e-86596ed3e55b None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:05.506127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5376b-d32f-4e2f-991e-86596ed3e55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:05.536480 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-4af5376b-d32f-4e2f-991e-86596ed3e55b None None] MFA Rules not processed for user `aff5e804c64345b298c745287a1d0a32`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:50:05.537644 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-4af5376b-d32f-4e2f-991e-86596ed3e55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:05.554365 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1081/2153] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 10 22:50:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:50:05.575432 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69774f0-2a2f-4ee8-8de4-5aeb4218085c service swift] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:05.587515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69774f0-2a2f-4ee8-8de4-5aeb4218085c service swift] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:05.594198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d5536-8343-469a-9f40-bd6b17cff735 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e3a76df95a34b9fb9816f22b85d1071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:05.594898 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d5536-8343-469a-9f40-bd6b17cff735 service swift] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:05.595432 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d5536-8343-469a-9f40-bd6b17cff735 service swift] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:05.595785 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d5536-8343-469a-9f40-bd6b17cff735 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:05.606369 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51d5536-8343-469a-9f40-bd6b17cff735 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:05.609054 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51d5536-8343-469a-9f40-bd6b17cff735 service swift] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:05.618990 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1073/2154] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 10 22:50:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 22:50:06.001005 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e3eaa3-3a05-4beb-9831-330f44f4b6d5 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:06.001397 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e3eaa3-3a05-4beb-9831-330f44f4b6d5 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:06.001776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e3eaa3-3a05-4beb-9831-330f44f4b6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:06.021196 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-a9e3eaa3-3a05-4beb-9831-330f44f4b6d5 None None] MFA Rules not processed for user `aff5e804c64345b298c745287a1d0a32`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:50:06.022754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-a9e3eaa3-3a05-4beb-9831-330f44f4b6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:06.030669 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1082/2155] 10.222.0.76 () {58 vars in 967 bytes} [Wed Aug 10 22:50:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:50:06.043285 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d5536-8343-469a-9f40-bd6b17cff735 service swift] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:06.057703 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d5536-8343-469a-9f40-bd6b17cff735 service swift] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:06.081624 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91076ad-a87a-4e68-8625-1a64a47c2d2c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e3a76df95a34b9fb9816f22b85d1071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:06.082335 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91076ad-a87a-4e68-8625-1a64a47c2d2c service swift] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:06.082695 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91076ad-a87a-4e68-8625-1a64a47c2d2c service swift] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:06.086901 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91076ad-a87a-4e68-8625-1a64a47c2d2c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:06.105112 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91076ad-a87a-4e68-8625-1a64a47c2d2c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:06.106950 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91076ad-a87a-4e68-8625-1a64a47c2d2c service swift] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:06.131523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1074/2156] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 10 22:50:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 22:50:06.508859 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e3eaa3-3a05-4beb-9831-330f44f4b6d5 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:06.597268 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e3eaa3-3a05-4beb-9831-330f44f4b6d5 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:06.602005 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4afd45b-e0fb-483a-8d34-74237aabdca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:06.602646 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afd45b-e0fb-483a-8d34-74237aabdca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:06.602926 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afd45b-e0fb-483a-8d34-74237aabdca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:06.603337 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afd45b-e0fb-483a-8d34-74237aabdca3 admin admin] PATH_INFO: `/v3/users/08701e6ef90949a3a478307944db381a` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:06.604477 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4afd45b-e0fb-483a-8d34-74237aabdca3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08701e6ef90949a3a478307944db381a)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:06.606695 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4afd45b-e0fb-483a-8d34-74237aabdca3 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:06.662888 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-e4afd45b-e0fb-483a-8d34-74237aabdca3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08701e6ef90949a3a478307944db381a'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:06.681848 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-e4afd45b-e0fb-483a-8d34-74237aabdca3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08701e6ef90949a3a478307944db381a'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:06.693434 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-e4afd45b-e0fb-483a-8d34-74237aabdca3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08701e6ef90949a3a478307944db381a'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:06.694744 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-e4afd45b-e0fb-483a-8d34-74237aabdca3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08701e6ef90949a3a478307944db381a'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:06.696255 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-e4afd45b-e0fb-483a-8d34-74237aabdca3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08701e6ef90949a3a478307944db381a'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:06.738433 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1083/2157] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:50:06 2022] DELETE /identity/v3/users/08701e6ef90949a3a478307944db381a => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:50:07.038306 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91076ad-a87a-4e68-8625-1a64a47c2d2c service swift] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:07.052933 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91076ad-a87a-4e68-8625-1a64a47c2d2c service swift] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:07.116982 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e0814f-5806-48fe-94d3-bd5e16dc152c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:07.117641 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0814f-5806-48fe-94d3-bd5e16dc152c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:07.117920 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0814f-5806-48fe-94d3-bd5e16dc152c admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:07.118192 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0814f-5806-48fe-94d3-bd5e16dc152c admin admin] PATH_INFO: `/v3/projects/2955b27b9b5748b3b51d4a7019bb7f81` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:07.119296 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e0814f-5806-48fe-94d3-bd5e16dc152c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2955b27b9b5748b3b51d4a7019bb7f81)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:07.121220 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e0814f-5806-48fe-94d3-bd5e16dc152c admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:07.170276 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-a8e0814f-5806-48fe-94d3-bd5e16dc152c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2955b27b9b5748b3b51d4a7019bb7f81', 'request_id': 'req-a8e0814f-5806-48fe-94d3-bd5e16dc152c'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:07.173149 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:50:07.179772 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-a8e0814f-5806-48fe-94d3-bd5e16dc152c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2955b27b9b5748b3b51d4a7019bb7f81', 'request_id': 'req-a8e0814f-5806-48fe-94d3-bd5e16dc152c'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:07.187564 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-a8e0814f-5806-48fe-94d3-bd5e16dc152c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2955b27b9b5748b3b51d4a7019bb7f81', 'request_id': 'req-a8e0814f-5806-48fe-94d3-bd5e16dc152c'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:07.201447 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-a8e0814f-5806-48fe-94d3-bd5e16dc152c admin admin] The token cache is being invalidate because project 2955b27b9b5748b3b51d4a7019bb7f81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:50:07.202560 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-a8e0814f-5806-48fe-94d3-bd5e16dc152c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:07.203350 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-a8e0814f-5806-48fe-94d3-bd5e16dc152c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:07.214182 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1075/2158] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:50:07 2022] DELETE /identity/v3/projects/2955b27b9b5748b3b51d4a7019bb7f81 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:50:07.214346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4afd45b-e0fb-483a-8d34-74237aabdca3 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:07.214346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-e4afd45b-e0fb-483a-8d34-74237aabdca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:07.228296 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:50:07.232716 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4afd45b-e0fb-483a-8d34-74237aabdca3 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:07.329962 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b2999-f493-419b-8b43-f317d1bb61a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:07.330989 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b2999-f493-419b-8b43-f317d1bb61a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:07.331515 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b2999-f493-419b-8b43-f317d1bb61a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:07.332002 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b2999-f493-419b-8b43-f317d1bb61a7 admin admin] PATH_INFO: `/v3/users/042bd1e4ec78487390f04fed12b0c1c1` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:07.333669 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00b2999-f493-419b-8b43-f317d1bb61a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=042bd1e4ec78487390f04fed12b0c1c1)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:07.336047 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00b2999-f493-419b-8b43-f317d1bb61a7 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:07.410005 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-f00b2999-f493-419b-8b43-f317d1bb61a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '042bd1e4ec78487390f04fed12b0c1c1'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:07.426107 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-f00b2999-f493-419b-8b43-f317d1bb61a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '042bd1e4ec78487390f04fed12b0c1c1'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:07.438724 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-f00b2999-f493-419b-8b43-f317d1bb61a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '042bd1e4ec78487390f04fed12b0c1c1'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:07.439981 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-f00b2999-f493-419b-8b43-f317d1bb61a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '042bd1e4ec78487390f04fed12b0c1c1'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:07.440635 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-f00b2999-f493-419b-8b43-f317d1bb61a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '042bd1e4ec78487390f04fed12b0c1c1'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:07.462451 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1084/2159] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:50:07 2022] DELETE /identity/v3/users/042bd1e4ec78487390f04fed12b0c1c1 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:50:07.685741 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e0814f-5806-48fe-94d3-bd5e16dc152c admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:07.688881 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-a8e0814f-5806-48fe-94d3-bd5e16dc152c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:07.711446 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e0814f-5806-48fe-94d3-bd5e16dc152c admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:07.776247 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14fa1b-306c-4101-8192-499b9ad55a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:07.776880 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14fa1b-306c-4101-8192-499b9ad55a34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:07.777157 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14fa1b-306c-4101-8192-499b9ad55a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:07.777466 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14fa1b-306c-4101-8192-499b9ad55a34 admin admin] PATH_INFO: `/v3/projects/5e536c3561a64d45be2a352cd5ee7b9a` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:07.778581 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b14fa1b-306c-4101-8192-499b9ad55a34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e536c3561a64d45be2a352cd5ee7b9a)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:07.781125 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b14fa1b-306c-4101-8192-499b9ad55a34 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:07.836275 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-7b14fa1b-306c-4101-8192-499b9ad55a34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e536c3561a64d45be2a352cd5ee7b9a', 'request_id': 'req-7b14fa1b-306c-4101-8192-499b9ad55a34'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:07.845976 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-7b14fa1b-306c-4101-8192-499b9ad55a34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e536c3561a64d45be2a352cd5ee7b9a', 'request_id': 'req-7b14fa1b-306c-4101-8192-499b9ad55a34'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:07.853594 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-7b14fa1b-306c-4101-8192-499b9ad55a34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e536c3561a64d45be2a352cd5ee7b9a', 'request_id': 'req-7b14fa1b-306c-4101-8192-499b9ad55a34'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:07.865894 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-7b14fa1b-306c-4101-8192-499b9ad55a34 admin admin] The token cache is being invalidate because project 5e536c3561a64d45be2a352cd5ee7b9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:50:07.866176 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-7b14fa1b-306c-4101-8192-499b9ad55a34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:07.866835 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-7b14fa1b-306c-4101-8192-499b9ad55a34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:07.868341 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1076/2160] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:50:07 2022] DELETE /identity/v3/projects/5e536c3561a64d45be2a352cd5ee7b9a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:50:07.874197 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795664e-6d38-4940-87a5-a3810e93b0c7 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:07.874646 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795664e-6d38-4940-87a5-a3810e93b0c7 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:07.874919 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795664e-6d38-4940-87a5-a3810e93b0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:07.894657 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-9795664e-6d38-4940-87a5-a3810e93b0c7 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:50:07.895873 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-9795664e-6d38-4940-87a5-a3810e93b0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:07.957120 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1085/2161] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:50:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:50:07.965116 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d191f3b-f5dc-4f9f-8516-2e173812c036 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:07.965469 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d191f3b-f5dc-4f9f-8516-2e173812c036 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:07.965745 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d191f3b-f5dc-4f9f-8516-2e173812c036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:07.986353 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-7d191f3b-f5dc-4f9f-8516-2e173812c036 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:50:07.987546 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-7d191f3b-f5dc-4f9f-8516-2e173812c036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:07.995453 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1077/2162] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:50:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:50:08.000518 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795664e-6d38-4940-87a5-a3810e93b0c7 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:08.010355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795664e-6d38-4940-87a5-a3810e93b0c7 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:08.015040 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7bbf7-0782-4f41-90f7-58f113610b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:08.015663 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7bbf7-0782-4f41-90f7-58f113610b4d admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:08.015946 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7bbf7-0782-4f41-90f7-58f113610b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:08.016221 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7bbf7-0782-4f41-90f7-58f113610b4d admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:08.016886 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d7bbf7-0782-4f41-90f7-58f113610b4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:50:08.017191 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d7bbf7-0782-4f41-90f7-58f113610b4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:08.018834 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d7bbf7-0782-4f41-90f7-58f113610b4d admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:08.030102 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1086/2163] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:50:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:50:08.034409 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d191f3b-f5dc-4f9f-8516-2e173812c036 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:08.043781 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d191f3b-f5dc-4f9f-8516-2e173812c036 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:08.049353 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b98be-69df-43dd-8652-cb1e5129e4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:08.049949 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b98be-69df-43dd-8652-cb1e5129e4d3 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:08.050231 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b98be-69df-43dd-8652-cb1e5129e4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:08.050507 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b98be-69df-43dd-8652-cb1e5129e4d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:08.051125 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b98be-69df-43dd-8652-cb1e5129e4d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:08.053030 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b98be-69df-43dd-8652-cb1e5129e4d3 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:08.081641 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1078/2164] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:50:08 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:50:08.085730 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7bbf7-0782-4f41-90f7-58f113610b4d admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:08.096630 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7bbf7-0782-4f41-90f7-58f113610b4d admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:08.104796 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:50:08.156815 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbc801-021f-4de4-8597-9d914c3c7482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:08.157512 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbc801-021f-4de4-8597-9d914c3c7482 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:08.157796 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbc801-021f-4de4-8597-9d914c3c7482 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:08.158072 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbc801-021f-4de4-8597-9d914c3c7482 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:08.158699 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bbc801-021f-4de4-8597-9d914c3c7482 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:08.160619 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bbc801-021f-4de4-8597-9d914c3c7482 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:08.181411 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1087/2165] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:50:08 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:50:08.185465 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b98be-69df-43dd-8652-cb1e5129e4d3 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:08.195721 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b98be-69df-43dd-8652-cb1e5129e4d3 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:08.200357 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f0670-19c9-4edb-92ca-b2ff8fb829b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:08.200932 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f0670-19c9-4edb-92ca-b2ff8fb829b9 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:08.201208 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f0670-19c9-4edb-92ca-b2ff8fb829b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:08.201547 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f0670-19c9-4edb-92ca-b2ff8fb829b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:08.202324 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f0670-19c9-4edb-92ca-b2ff8fb829b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:08.203948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f0670-19c9-4edb-92ca-b2ff8fb829b9 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:08.213918 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1079/2166] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:50:08.218469 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbc801-021f-4de4-8597-9d914c3c7482 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:08.228526 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbc801-021f-4de4-8597-9d914c3c7482 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:08.233892 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f7a162-8710-479c-aade-f0dd57389d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:08.234501 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7a162-8710-479c-aade-f0dd57389d86 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:08.234786 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7a162-8710-479c-aade-f0dd57389d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:08.235056 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7a162-8710-479c-aade-f0dd57389d86 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:08.235698 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f7a162-8710-479c-aade-f0dd57389d86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:08.237410 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f7a162-8710-479c-aade-f0dd57389d86 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:08.247577 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1088/2167] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:50:08.251840 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f0670-19c9-4edb-92ca-b2ff8fb829b9 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:08.262171 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f0670-19c9-4edb-92ca-b2ff8fb829b9 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:08.267058 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84eb1f0-d964-4079-a31b-40166dabf3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:08.267743 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84eb1f0-d964-4079-a31b-40166dabf3ab admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:08.268026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84eb1f0-d964-4079-a31b-40166dabf3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:08.268302 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84eb1f0-d964-4079-a31b-40166dabf3ab admin admin] PATH_INFO: `/v3/projects/6e9d10d60f3b4918a3dc554d93c86d7b/users/84b8d41c742d40499ef0a719e1211097/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:08.283481 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84eb1f0-d964-4079-a31b-40166dabf3ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e9d10d60f3b4918a3dc554d93c86d7b, user_id=84b8d41c742d40499ef0a719e1211097, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:08.286200 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84eb1f0-d964-4079-a31b-40166dabf3ab admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:08.299765 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1080/2168] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:50:08 2022] PUT /identity/v3/projects/6e9d10d60f3b4918a3dc554d93c86d7b/users/84b8d41c742d40499ef0a719e1211097/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:50:08.310070 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b1c53-4af7-41e4-b616-b0deb75422b2 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:08.310356 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b1c53-4af7-41e4-b616-b0deb75422b2 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:08.310629 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b1c53-4af7-41e4-b616-b0deb75422b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:08.338439 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-3f7b1c53-4af7-41e4-b616-b0deb75422b2 None None] MFA Rules not processed for user `84b8d41c742d40499ef0a719e1211097`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:50:08.339649 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-3f7b1c53-4af7-41e4-b616-b0deb75422b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:08.404134 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1089/2169] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:50:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:50:08.415668 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84eb1f0-d964-4079-a31b-40166dabf3ab admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:08.417497 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-c84eb1f0-d964-4079-a31b-40166dabf3ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:08.428599 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:50:08.480200 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84eb1f0-d964-4079-a31b-40166dabf3ab admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:08.484698 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f95dfe2-5fbf-4868-be5f-692ca8cc6022 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc7524edf5d5422f91d92fd91be5ee1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:08.485376 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95dfe2-5fbf-4868-be5f-692ca8cc6022 service glance] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:08.485656 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95dfe2-5fbf-4868-be5f-692ca8cc6022 service glance] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:08.485927 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95dfe2-5fbf-4868-be5f-692ca8cc6022 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:08.494723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f95dfe2-5fbf-4868-be5f-692ca8cc6022 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:08.496145 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f95dfe2-5fbf-4868-be5f-692ca8cc6022 service glance] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:08.505098 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1081/2170] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:50:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:50:08.554171 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b1c53-4af7-41e4-b616-b0deb75422b2 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:08.555591 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-3f7b1c53-4af7-41e4-b616-b0deb75422b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:08.623192 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b1c53-4af7-41e4-b616-b0deb75422b2 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:08.629640 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f922ea9-4346-430d-bdde-9aecfaa6c977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:08.630377 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f922ea9-4346-430d-bdde-9aecfaa6c977 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:08.630661 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f922ea9-4346-430d-bdde-9aecfaa6c977 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:08.630930 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f922ea9-4346-430d-bdde-9aecfaa6c977 admin admin] PATH_INFO: `/v3/users/e15f5f4d1ca74dba89d80dfa5ca87563` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:08.632081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f922ea9-4346-430d-bdde-9aecfaa6c977 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e15f5f4d1ca74dba89d80dfa5ca87563)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:08.634193 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f922ea9-4346-430d-bdde-9aecfaa6c977 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:08.679380 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:50:08.718137 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-3f922ea9-4346-430d-bdde-9aecfaa6c977 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e15f5f4d1ca74dba89d80dfa5ca87563'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:08.735469 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-3f922ea9-4346-430d-bdde-9aecfaa6c977 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e15f5f4d1ca74dba89d80dfa5ca87563'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:08.757734 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-3f922ea9-4346-430d-bdde-9aecfaa6c977 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e15f5f4d1ca74dba89d80dfa5ca87563'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:08.758502 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-3f922ea9-4346-430d-bdde-9aecfaa6c977 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e15f5f4d1ca74dba89d80dfa5ca87563'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:08.759274 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-3f922ea9-4346-430d-bdde-9aecfaa6c977 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e15f5f4d1ca74dba89d80dfa5ca87563'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:08.789505 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1090/2171] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:50:08 2022] DELETE /identity/v3/users/e15f5f4d1ca74dba89d80dfa5ca87563 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:50:08.794105 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e0ed0-089b-4b2f-87c4-f2f576e927d3 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:08.794561 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e0ed0-089b-4b2f-87c4-f2f576e927d3 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:08.794839 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e0ed0-089b-4b2f-87c4-f2f576e927d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:08.822078 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-2d9e0ed0-089b-4b2f-87c4-f2f576e927d3 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:50:08.826146 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-2d9e0ed0-089b-4b2f-87c4-f2f576e927d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:08.935208 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1082/2172] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:50:08.937124 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f922ea9-4346-430d-bdde-9aecfaa6c977 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:08.938664 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-3f922ea9-4346-430d-bdde-9aecfaa6c977 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:09.009749 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f922ea9-4346-430d-bdde-9aecfaa6c977 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:09.014834 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69736bdf-25d3-41de-837b-0089f7f02526 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:09.015723 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69736bdf-25d3-41de-837b-0089f7f02526 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:09.018052 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69736bdf-25d3-41de-837b-0089f7f02526 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:09.018052 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69736bdf-25d3-41de-837b-0089f7f02526 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:09.037412 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69736bdf-25d3-41de-837b-0089f7f02526 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:09.038927 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69736bdf-25d3-41de-837b-0089f7f02526 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:09.052574 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1091/2173] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:50:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:50:09.272349 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e0ed0-089b-4b2f-87c4-f2f576e927d3 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:09.273931 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-2d9e0ed0-089b-4b2f-87c4-f2f576e927d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:09.293519 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e0ed0-089b-4b2f-87c4-f2f576e927d3 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:09.308396 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38313803-a68c-4b4f-aba3-b70fb8641b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:09.309273 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38313803-a68c-4b4f-aba3-b70fb8641b8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:09.309803 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38313803-a68c-4b4f-aba3-b70fb8641b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:09.313250 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38313803-a68c-4b4f-aba3-b70fb8641b8a admin admin] PATH_INFO: `/v3/projects/dcf2ffea147f4700b67bfc7accdd0815` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:09.314635 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38313803-a68c-4b4f-aba3-b70fb8641b8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcf2ffea147f4700b67bfc7accdd0815)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:09.316592 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38313803-a68c-4b4f-aba3-b70fb8641b8a admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:09.392221 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-38313803-a68c-4b4f-aba3-b70fb8641b8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcf2ffea147f4700b67bfc7accdd0815', 'request_id': 'req-38313803-a68c-4b4f-aba3-b70fb8641b8a'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:09.402594 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-38313803-a68c-4b4f-aba3-b70fb8641b8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcf2ffea147f4700b67bfc7accdd0815', 'request_id': 'req-38313803-a68c-4b4f-aba3-b70fb8641b8a'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:09.409523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-38313803-a68c-4b4f-aba3-b70fb8641b8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcf2ffea147f4700b67bfc7accdd0815', 'request_id': 'req-38313803-a68c-4b4f-aba3-b70fb8641b8a'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:09.425605 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-38313803-a68c-4b4f-aba3-b70fb8641b8a admin admin] The token cache is being invalidate because project dcf2ffea147f4700b67bfc7accdd0815 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:50:09.425605 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-38313803-a68c-4b4f-aba3-b70fb8641b8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:09.426219 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-38313803-a68c-4b4f-aba3-b70fb8641b8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:09.427732 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1083/2174] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:50:09 2022] DELETE /identity/v3/projects/dcf2ffea147f4700b67bfc7accdd0815 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:50:09.432817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe516c-7bf2-4940-b288-c59642794d85 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:09.433693 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe516c-7bf2-4940-b288-c59642794d85 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:09.433693 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe516c-7bf2-4940-b288-c59642794d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:09.459397 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-ecfe516c-7bf2-4940-b288-c59642794d85 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:50:09.461599 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-ecfe516c-7bf2-4940-b288-c59642794d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:09.541730 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1092/2175] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:50:09.547831 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed4825-9b9a-40c2-851c-48006b835b5f None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:09.548111 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed4825-9b9a-40c2-851c-48006b835b5f None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:09.548384 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed4825-9b9a-40c2-851c-48006b835b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:09.569509 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-a7ed4825-9b9a-40c2-851c-48006b835b5f None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:50:09.570766 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-a7ed4825-9b9a-40c2-851c-48006b835b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:09.579030 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1084/2176] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:50:09.583450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe516c-7bf2-4940-b288-c59642794d85 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:09.593949 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe516c-7bf2-4940-b288-c59642794d85 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:09.598568 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ae5b2-1c9d-4ad0-a691-a124fb5c9f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:09.599178 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ae5b2-1c9d-4ad0-a691-a124fb5c9f68 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:09.599461 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ae5b2-1c9d-4ad0-a691-a124fb5c9f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:09.599738 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ae5b2-1c9d-4ad0-a691-a124fb5c9f68 admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:09.600391 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ae5b2-1c9d-4ad0-a691-a124fb5c9f68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:50:09.600697 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ae5b2-1c9d-4ad0-a691-a124fb5c9f68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:09.602346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ae5b2-1c9d-4ad0-a691-a124fb5c9f68 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:09.615471 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1093/2177] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:50:09.619813 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed4825-9b9a-40c2-851c-48006b835b5f None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:09.629599 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed4825-9b9a-40c2-851c-48006b835b5f None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:09.634318 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ffe8f2-4d1e-4017-ae9a-9e7a93a29fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:09.635008 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffe8f2-4d1e-4017-ae9a-9e7a93a29fdd admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:09.635282 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffe8f2-4d1e-4017-ae9a-9e7a93a29fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:09.635550 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffe8f2-4d1e-4017-ae9a-9e7a93a29fdd admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:09.636171 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ffe8f2-4d1e-4017-ae9a-9e7a93a29fdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:09.638139 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ffe8f2-4d1e-4017-ae9a-9e7a93a29fdd admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:09.669403 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1085/2178] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:50:09 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:50:09.672304 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ae5b2-1c9d-4ad0-a691-a124fb5c9f68 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:09.686506 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ae5b2-1c9d-4ad0-a691-a124fb5c9f68 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:09.715298 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:50:09.756104 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b3c427-b375-4984-b42d-33b9502d4d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:09.756789 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3c427-b375-4984-b42d-33b9502d4d11 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:09.757143 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3c427-b375-4984-b42d-33b9502d4d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:09.757544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3c427-b375-4984-b42d-33b9502d4d11 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:09.758255 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b3c427-b375-4984-b42d-33b9502d4d11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:09.760236 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b3c427-b375-4984-b42d-33b9502d4d11 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:09.787041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1094/2179] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:50:09 2022] POST /identity/v3/users => generated 291 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:50:09.788266 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ffe8f2-4d1e-4017-ae9a-9e7a93a29fdd admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:09.794862 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:50:09.800265 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ffe8f2-4d1e-4017-ae9a-9e7a93a29fdd admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:09.806847 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a81f8b-d674-4f5c-a7fd-9f85057260e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:09.807485 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a81f8b-d674-4f5c-a7fd-9f85057260e9 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:09.808073 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a81f8b-d674-4f5c-a7fd-9f85057260e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:09.808349 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a81f8b-d674-4f5c-a7fd-9f85057260e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:09.809168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a81f8b-d674-4f5c-a7fd-9f85057260e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:09.811715 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a81f8b-d674-4f5c-a7fd-9f85057260e9 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:09.822666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1086/2180] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:50:09.831204 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b3c427-b375-4984-b42d-33b9502d4d11 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:09.838001 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b3c427-b375-4984-b42d-33b9502d4d11 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:09.843738 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4d4c8-6409-428d-9129-9e4020becb44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:09.844354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4d4c8-6409-428d-9129-9e4020becb44 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:09.844648 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4d4c8-6409-428d-9129-9e4020becb44 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:09.844908 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4d4c8-6409-428d-9129-9e4020becb44 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:09.845570 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b4d4c8-6409-428d-9129-9e4020becb44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:09.853011 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b4d4c8-6409-428d-9129-9e4020becb44 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:09.863765 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1095/2181] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 22:50:09.869581 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a81f8b-d674-4f5c-a7fd-9f85057260e9 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:09.881536 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a81f8b-d674-4f5c-a7fd-9f85057260e9 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:09.894704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4247171c-1860-430f-bb76-be00ca928877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:09.895394 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4247171c-1860-430f-bb76-be00ca928877 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:09.895677 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4247171c-1860-430f-bb76-be00ca928877 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:09.895954 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4247171c-1860-430f-bb76-be00ca928877 admin admin] PATH_INFO: `/v3/projects/0f4c72696c814e6e816fd9240569e0db/users/b9f5b7c7c05c42eda7926cd03a0fd9bc/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:09.911416 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4247171c-1860-430f-bb76-be00ca928877 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f4c72696c814e6e816fd9240569e0db, user_id=b9f5b7c7c05c42eda7926cd03a0fd9bc, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:09.914438 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4247171c-1860-430f-bb76-be00ca928877 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:09.929663 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1087/2182] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:50:09 2022] PUT /identity/v3/projects/0f4c72696c814e6e816fd9240569e0db/users/b9f5b7c7c05c42eda7926cd03a0fd9bc/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:50:09.934665 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b985fc-5c79-45e3-ba72-1aee4780ca59 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:09.934952 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b985fc-5c79-45e3-ba72-1aee4780ca59 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:09.935229 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b985fc-5c79-45e3-ba72-1aee4780ca59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:09.957363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-33b985fc-5c79-45e3-ba72-1aee4780ca59 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:50:09.958703 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-33b985fc-5c79-45e3-ba72-1aee4780ca59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:10.036815 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1096/2183] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:50:10.049317 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4247171c-1860-430f-bb76-be00ca928877 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:10.051118 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-4247171c-1860-430f-bb76-be00ca928877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:10.112805 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4247171c-1860-430f-bb76-be00ca928877 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:10.116670 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da60de4a-63a3-4f16-9b59-8c7937ee61ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:10.117260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da60de4a-63a3-4f16-9b59-8c7937ee61ef service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:10.117572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da60de4a-63a3-4f16-9b59-8c7937ee61ef service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:10.117847 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da60de4a-63a3-4f16-9b59-8c7937ee61ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:10.130556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da60de4a-63a3-4f16-9b59-8c7937ee61ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:10.132718 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da60de4a-63a3-4f16-9b59-8c7937ee61ef service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:10.146288 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1088/2184] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:50:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:50:14.585109 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97d76e-e3ed-46f4-b268-59e6336cec38 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:14.585666 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97d76e-e3ed-46f4-b268-59e6336cec38 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:14.585666 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97d76e-e3ed-46f4-b268-59e6336cec38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:14.620461 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-4b97d76e-e3ed-46f4-b268-59e6336cec38 None None] MFA Rules not processed for user `b9f5b7c7c05c42eda7926cd03a0fd9bc`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:50:14.621734 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-4b97d76e-e3ed-46f4-b268-59e6336cec38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:14.700859 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1097/2185] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:50:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:50:14.705930 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da60de4a-63a3-4f16-9b59-8c7937ee61ef service neutron] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:14.716439 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da60de4a-63a3-4f16-9b59-8c7937ee61ef service neutron] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:14.722786 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb489920-5e36-4ee7-a016-8f90eb17f05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:14.723414 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb489920-5e36-4ee7-a016-8f90eb17f05c admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:14.723693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb489920-5e36-4ee7-a016-8f90eb17f05c admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:14.724088 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb489920-5e36-4ee7-a016-8f90eb17f05c admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:14.724584 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb489920-5e36-4ee7-a016-8f90eb17f05c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:14.726583 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb489920-5e36-4ee7-a016-8f90eb17f05c admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:14.759420 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1089/2186] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:50:14 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:50:14.764098 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b97d76e-e3ed-46f4-b268-59e6336cec38 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:14.776601 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b97d76e-e3ed-46f4-b268-59e6336cec38 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:14.821876 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:50:14.850423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50f080-cff2-4adf-962c-78a85aa0b458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:14.851049 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50f080-cff2-4adf-962c-78a85aa0b458 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:14.851701 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50f080-cff2-4adf-962c-78a85aa0b458 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:14.851701 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50f080-cff2-4adf-962c-78a85aa0b458 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:14.852201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50f080-cff2-4adf-962c-78a85aa0b458 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:14.854345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50f080-cff2-4adf-962c-78a85aa0b458 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:14.884027 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1098/2187] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:50:14 2022] POST /identity/v3/users => generated 298 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:50:14.889223 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb489920-5e36-4ee7-a016-8f90eb17f05c admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:14.900934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb489920-5e36-4ee7-a016-8f90eb17f05c admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:14.911777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe364651-4c5e-4007-9ca9-574e94c1239a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:14.912939 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe364651-4c5e-4007-9ca9-574e94c1239a admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:14.913343 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe364651-4c5e-4007-9ca9-574e94c1239a admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:14.913706 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe364651-4c5e-4007-9ca9-574e94c1239a admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:14.914443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe364651-4c5e-4007-9ca9-574e94c1239a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:14.916470 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe364651-4c5e-4007-9ca9-574e94c1239a admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:14.931593 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1090/2188] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:50:14.936609 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50f080-cff2-4adf-962c-78a85aa0b458 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:14.956332 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50f080-cff2-4adf-962c-78a85aa0b458 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:14.965602 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f858e978-a872-4ab5-b71d-42761a82522a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:14.966380 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f858e978-a872-4ab5-b71d-42761a82522a admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:14.966769 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f858e978-a872-4ab5-b71d-42761a82522a admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:14.967320 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f858e978-a872-4ab5-b71d-42761a82522a admin admin] PATH_INFO: `/v3/projects/bea0c702d60148f5b6f225f16c760e71/users/988cb2ebefcd4620b8bbdc6d0647dc7f/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:14.985116 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f858e978-a872-4ab5-b71d-42761a82522a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bea0c702d60148f5b6f225f16c760e71, user_id=988cb2ebefcd4620b8bbdc6d0647dc7f, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:14.987927 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f858e978-a872-4ab5-b71d-42761a82522a admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:15.007716 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1099/2189] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:50:14 2022] PUT /identity/v3/projects/bea0c702d60148f5b6f225f16c760e71/users/988cb2ebefcd4620b8bbdc6d0647dc7f/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:50:15.011636 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe364651-4c5e-4007-9ca9-574e94c1239a admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:15.033215 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe364651-4c5e-4007-9ca9-574e94c1239a admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:15.117857 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102feb5-f068-4466-92be-24b9d9e6f418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:15.118475 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102feb5-f068-4466-92be-24b9d9e6f418 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:15.118755 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102feb5-f068-4466-92be-24b9d9e6f418 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:15.119029 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102feb5-f068-4466-92be-24b9d9e6f418 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:15.119672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102feb5-f068-4466-92be-24b9d9e6f418 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:15.121346 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102feb5-f068-4466-92be-24b9d9e6f418 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:15.134123 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1091/2190] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 22:50:15.142013 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f858e978-a872-4ab5-b71d-42761a82522a admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:15.156651 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f858e978-a872-4ab5-b71d-42761a82522a admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:15.163222 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dafedd-6f6a-41f1-9c06-f7a5fb6f2d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:15.163941 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dafedd-6f6a-41f1-9c06-f7a5fb6f2d0f admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:15.164286 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dafedd-6f6a-41f1-9c06-f7a5fb6f2d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:15.164634 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dafedd-6f6a-41f1-9c06-f7a5fb6f2d0f admin admin] PATH_INFO: `/v3/projects/bea0c702d60148f5b6f225f16c760e71/users/988cb2ebefcd4620b8bbdc6d0647dc7f/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:15.167043 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dafedd-6f6a-41f1-9c06-f7a5fb6f2d0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bea0c702d60148f5b6f225f16c760e71, user_id=988cb2ebefcd4620b8bbdc6d0647dc7f, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:15.170592 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dafedd-6f6a-41f1-9c06-f7a5fb6f2d0f admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:15.183026 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1100/2191] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:50:15 2022] PUT /identity/v3/projects/bea0c702d60148f5b6f225f16c760e71/users/988cb2ebefcd4620b8bbdc6d0647dc7f/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:50:17.531521 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ebfa41-8c0a-486a-86d3-0e7629909f8f None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:17.532178 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ebfa41-8c0a-486a-86d3-0e7629909f8f None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:17.532178 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ebfa41-8c0a-486a-86d3-0e7629909f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:17.577760 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-b2ebfa41-8c0a-486a-86d3-0e7629909f8f None None] MFA Rules not processed for user `aff5e804c64345b298c745287a1d0a32`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:50:17.580565 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-b2ebfa41-8c0a-486a-86d3-0e7629909f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:17.689917 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:50:17.803121 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1092/2192] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 10 22:50:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:50:17.820309 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dafedd-6f6a-41f1-9c06-f7a5fb6f2d0f admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:17.823176 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-15dafedd-6f6a-41f1-9c06-f7a5fb6f2d0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:17.946798 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dafedd-6f6a-41f1-9c06-f7a5fb6f2d0f admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:17.956831 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ae9d9-ed30-4060-8fdc-339c8915ebd5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e3a76df95a34b9fb9816f22b85d1071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:17.957820 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ae9d9-ed30-4060-8fdc-339c8915ebd5 service swift] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:17.958254 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ae9d9-ed30-4060-8fdc-339c8915ebd5 service swift] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:17.961059 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ae9d9-ed30-4060-8fdc-339c8915ebd5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:17.976412 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ae9d9-ed30-4060-8fdc-339c8915ebd5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:17.978421 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ae9d9-ed30-4060-8fdc-339c8915ebd5 service swift] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:18.010393 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1101/2193] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 10 22:50:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 22:50:21.548084 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b6999-ecff-4fcf-8535-25502a00af22 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:21.549026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b6999-ecff-4fcf-8535-25502a00af22 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:21.549509 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b6999-ecff-4fcf-8535-25502a00af22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:21.609691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-ab1b6999-ecff-4fcf-8535-25502a00af22 None None] MFA Rules not processed for user `988cb2ebefcd4620b8bbdc6d0647dc7f`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:50:21.610968 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-ab1b6999-ecff-4fcf-8535-25502a00af22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:21.717865 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1093/2194] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:50:21 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:50:27.821492 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ae9d9-ed30-4060-8fdc-339c8915ebd5 service swift] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:27.822951 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-9d2ae9d9-ed30-4060-8fdc-339c8915ebd5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:27.893558 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ae9d9-ed30-4060-8fdc-339c8915ebd5 service swift] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:27.897461 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bafd769-a29e-41b3-97e0-1f5f9a626440 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc7524edf5d5422f91d92fd91be5ee1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:27.898067 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bafd769-a29e-41b3-97e0-1f5f9a626440 service glance] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:27.898373 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bafd769-a29e-41b3-97e0-1f5f9a626440 service glance] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:27.898644 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bafd769-a29e-41b3-97e0-1f5f9a626440 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:27.944782 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:50:27.976878 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bafd769-a29e-41b3-97e0-1f5f9a626440 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:27.978605 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bafd769-a29e-41b3-97e0-1f5f9a626440 service glance] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:27.990051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1102/2195] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:50:27 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:50:28.139731 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad4c67-b1eb-4fa6-a2fa-d5898f51d14b None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:28.140252 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad4c67-b1eb-4fa6-a2fa-d5898f51d14b None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:28.140334 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad4c67-b1eb-4fa6-a2fa-d5898f51d14b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:28.162147 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-e9ad4c67-b1eb-4fa6-a2fa-d5898f51d14b None None] MFA Rules not processed for user `aff5e804c64345b298c745287a1d0a32`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:50:28.163702 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-e9ad4c67-b1eb-4fa6-a2fa-d5898f51d14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:28.172413 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1094/2196] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 10 22:50:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:50:28.188159 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bafd769-a29e-41b3-97e0-1f5f9a626440 service glance] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:28.201504 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bafd769-a29e-41b3-97e0-1f5f9a626440 service glance] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:28.206886 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba63510-359c-4e14-8320-7dfa13932cca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e3a76df95a34b9fb9816f22b85d1071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:28.207588 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba63510-359c-4e14-8320-7dfa13932cca service swift] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:28.207872 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba63510-359c-4e14-8320-7dfa13932cca service swift] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:28.208142 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba63510-359c-4e14-8320-7dfa13932cca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:28.218237 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba63510-359c-4e14-8320-7dfa13932cca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:28.220059 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba63510-359c-4e14-8320-7dfa13932cca service swift] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:28.232052 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1103/2197] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 10 22:50:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 22:50:38.137793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd9d7b-3b03-4c3e-b81a-89634ee5bf35 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:38.138379 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd9d7b-3b03-4c3e-b81a-89634ee5bf35 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:38.138379 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd9d7b-3b03-4c3e-b81a-89634ee5bf35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:38.160070 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-88fd9d7b-3b03-4c3e-b81a-89634ee5bf35 None None] MFA Rules not processed for user `b9f5b7c7c05c42eda7926cd03a0fd9bc`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:50:38.161655 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-88fd9d7b-3b03-4c3e-b81a-89634ee5bf35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:38.173672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1095/2198] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:50:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:50:38.227737 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=18 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:50:39.495785 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba63510-359c-4e14-8320-7dfa13932cca service swift] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:39.497954 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-4ba63510-359c-4e14-8320-7dfa13932cca service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:39.589993 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba63510-359c-4e14-8320-7dfa13932cca service swift] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:39.597787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aeb3f6-e223-4933-8561-e08404b93888 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f46dee0f73e49eab54c11556e3b86bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:39.599177 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aeb3f6-e223-4933-8561-e08404b93888 service placement] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:39.599340 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aeb3f6-e223-4933-8561-e08404b93888 service placement] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:39.599588 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aeb3f6-e223-4933-8561-e08404b93888 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:39.601519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-d2aeb3f6-e223-4933-8561-e08404b93888 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:39.615410 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aeb3f6-e223-4933-8561-e08404b93888 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:39.616957 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aeb3f6-e223-4933-8561-e08404b93888 service placement] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:39.627856 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1104/2199] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:50:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:50:42.058931 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af4c82-1bec-42d9-b38a-b4b81ac2c571 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:42.059627 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af4c82-1bec-42d9-b38a-b4b81ac2c571 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:42.059862 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af4c82-1bec-42d9-b38a-b4b81ac2c571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:42.083750 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-d3af4c82-1bec-42d9-b38a-b4b81ac2c571 None None] MFA Rules not processed for user `aff5e804c64345b298c745287a1d0a32`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:50:42.084970 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-d3af4c82-1bec-42d9-b38a-b4b81ac2c571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:42.105366 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1096/2200] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 10 22:50:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:50:42.137214 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aeb3f6-e223-4933-8561-e08404b93888 service placement] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:42.161771 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aeb3f6-e223-4933-8561-e08404b93888 service placement] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:42.174899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcc59d6-dc65-4d29-89ef-93d0646f6ad6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e3a76df95a34b9fb9816f22b85d1071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:42.177595 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc59d6-dc65-4d29-89ef-93d0646f6ad6 service swift] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:42.178529 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc59d6-dc65-4d29-89ef-93d0646f6ad6 service swift] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:42.180107 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc59d6-dc65-4d29-89ef-93d0646f6ad6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:42.196705 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc59d6-dc65-4d29-89ef-93d0646f6ad6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:42.199250 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc59d6-dc65-4d29-89ef-93d0646f6ad6 service swift] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:42.226508 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1105/2201] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 10 22:50:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 22:50:44.458939 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef53ad-bd4e-4272-8e62-4023d2a77b4d None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:44.459443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef53ad-bd4e-4272-8e62-4023d2a77b4d None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:44.459556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef53ad-bd4e-4272-8e62-4023d2a77b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:44.478964 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-f8ef53ad-bd4e-4272-8e62-4023d2a77b4d None None] MFA Rules not processed for user `aff5e804c64345b298c745287a1d0a32`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:50:44.480196 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-f8ef53ad-bd4e-4272-8e62-4023d2a77b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:44.487273 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1097/2202] 10.222.0.76 () {58 vars in 967 bytes} [Wed Aug 10 22:50:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:50:44.500020 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcc59d6-dc65-4d29-89ef-93d0646f6ad6 service swift] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:44.510722 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcc59d6-dc65-4d29-89ef-93d0646f6ad6 service swift] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:44.516839 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c7f67-387d-4401-a134-82757a136a8d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e3a76df95a34b9fb9816f22b85d1071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:44.517482 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c7f67-387d-4401-a134-82757a136a8d service swift] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:44.517762 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c7f67-387d-4401-a134-82757a136a8d service swift] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:44.518424 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c7f67-387d-4401-a134-82757a136a8d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:44.527779 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7c7f67-387d-4401-a134-82757a136a8d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:44.529658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7c7f67-387d-4401-a134-82757a136a8d service swift] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:44.547748 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1106/2203] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 10 22:50:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 22:50:46.922094 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef53ad-bd4e-4272-8e62-4023d2a77b4d None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:46.924095 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-f8ef53ad-bd4e-4272-8e62-4023d2a77b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:47.000964 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef53ad-bd4e-4272-8e62-4023d2a77b4d None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:47.005435 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb77f74e-0c91-48ef-898d-45ef0a2bd734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:47.006067 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77f74e-0c91-48ef-898d-45ef0a2bd734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:47.006341 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77f74e-0c91-48ef-898d-45ef0a2bd734 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:47.006611 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77f74e-0c91-48ef-898d-45ef0a2bd734 admin admin] PATH_INFO: `/v3/users/0f645ba95eb64bb2b23d0d38a141307a` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:47.007747 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb77f74e-0c91-48ef-898d-45ef0a2bd734 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f645ba95eb64bb2b23d0d38a141307a)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:47.009717 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb77f74e-0c91-48ef-898d-45ef0a2bd734 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:47.062008 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:50:47.068435 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-eb77f74e-0c91-48ef-898d-45ef0a2bd734 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f645ba95eb64bb2b23d0d38a141307a'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:47.077033 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-eb77f74e-0c91-48ef-898d-45ef0a2bd734 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f645ba95eb64bb2b23d0d38a141307a'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:47.096261 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-eb77f74e-0c91-48ef-898d-45ef0a2bd734 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f645ba95eb64bb2b23d0d38a141307a'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:47.097083 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-eb77f74e-0c91-48ef-898d-45ef0a2bd734 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f645ba95eb64bb2b23d0d38a141307a'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:47.097770 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-eb77f74e-0c91-48ef-898d-45ef0a2bd734 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f645ba95eb64bb2b23d0d38a141307a'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:47.112164 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1098/2204] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:50:46 2022] DELETE /identity/v3/users/0f645ba95eb64bb2b23d0d38a141307a => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:50:47.296471 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c7f67-387d-4401-a134-82757a136a8d service swift] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:47.312763 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c7f67-387d-4401-a134-82757a136a8d service swift] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:47.390604 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d1a14-1916-44f8-b413-6788821bc254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:47.391410 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d1a14-1916-44f8-b413-6788821bc254 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:47.391792 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d1a14-1916-44f8-b413-6788821bc254 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:47.392170 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d1a14-1916-44f8-b413-6788821bc254 admin admin] PATH_INFO: `/v3/projects/50a0ee776f574f46bdb5f16c1b864402` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:47.393446 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d1a14-1916-44f8-b413-6788821bc254 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50a0ee776f574f46bdb5f16c1b864402)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:47.395478 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d1a14-1916-44f8-b413-6788821bc254 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:47.455475 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-d58d1a14-1916-44f8-b413-6788821bc254 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50a0ee776f574f46bdb5f16c1b864402', 'request_id': 'req-d58d1a14-1916-44f8-b413-6788821bc254'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:47.460913 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-d58d1a14-1916-44f8-b413-6788821bc254 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50a0ee776f574f46bdb5f16c1b864402', 'request_id': 'req-d58d1a14-1916-44f8-b413-6788821bc254'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:47.468916 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-d58d1a14-1916-44f8-b413-6788821bc254 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50a0ee776f574f46bdb5f16c1b864402', 'request_id': 'req-d58d1a14-1916-44f8-b413-6788821bc254'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:47.470140 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:50:47.482986 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-d58d1a14-1916-44f8-b413-6788821bc254 admin admin] The token cache is being invalidate because project 50a0ee776f574f46bdb5f16c1b864402 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:50:47.483280 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-d58d1a14-1916-44f8-b413-6788821bc254 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:47.483947 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-d58d1a14-1916-44f8-b413-6788821bc254 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:47.485485 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1107/2205] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:50:47 2022] DELETE /identity/v3/projects/50a0ee776f574f46bdb5f16c1b864402 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:50:47.490496 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3374b8be-a937-4f78-a8e4-7bbd7c0b8be2 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:47.490781 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3374b8be-a937-4f78-a8e4-7bbd7c0b8be2 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:47.491053 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3374b8be-a937-4f78-a8e4-7bbd7c0b8be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:47.513245 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-3374b8be-a937-4f78-a8e4-7bbd7c0b8be2 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:50:47.516063 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-3374b8be-a937-4f78-a8e4-7bbd7c0b8be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:47.587831 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1099/2206] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:50:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:50:47.594012 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be3ae4-861b-4e87-8023-17b93d4c2eac None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:47.594307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be3ae4-861b-4e87-8023-17b93d4c2eac None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:47.594587 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be3ae4-861b-4e87-8023-17b93d4c2eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:47.615861 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-d9be3ae4-861b-4e87-8023-17b93d4c2eac None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:50:47.617318 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-d9be3ae4-861b-4e87-8023-17b93d4c2eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:47.625716 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1108/2207] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:50:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:50:47.630718 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3374b8be-a937-4f78-a8e4-7bbd7c0b8be2 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:47.642538 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3374b8be-a937-4f78-a8e4-7bbd7c0b8be2 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:47.653344 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a09dc89-f7e6-4cff-80e8-36d4f0c8c9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:47.653344 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09dc89-f7e6-4cff-80e8-36d4f0c8c9ab admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:47.657425 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09dc89-f7e6-4cff-80e8-36d4f0c8c9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:47.657951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09dc89-f7e6-4cff-80e8-36d4f0c8c9ab admin admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:47.658984 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a09dc89-f7e6-4cff-80e8-36d4f0c8c9ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:50:47.659437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a09dc89-f7e6-4cff-80e8-36d4f0c8c9ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:47.661242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a09dc89-f7e6-4cff-80e8-36d4f0c8c9ab admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:47.675749 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1100/2208] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:50:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:50:47.680169 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be3ae4-861b-4e87-8023-17b93d4c2eac None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:47.689480 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be3ae4-861b-4e87-8023-17b93d4c2eac None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:47.694246 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224fb13-859d-45ef-a9f6-3b6ab20cf9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:47.694857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224fb13-859d-45ef-a9f6-3b6ab20cf9aa admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:47.695134 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224fb13-859d-45ef-a9f6-3b6ab20cf9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:47.695404 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224fb13-859d-45ef-a9f6-3b6ab20cf9aa admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:47.696034 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224fb13-859d-45ef-a9f6-3b6ab20cf9aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:47.697972 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224fb13-859d-45ef-a9f6-3b6ab20cf9aa admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:47.729075 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1109/2209] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:50:47 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:50:47.733704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a09dc89-f7e6-4cff-80e8-36d4f0c8c9ab admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:47.744575 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a09dc89-f7e6-4cff-80e8-36d4f0c8c9ab admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:47.815257 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c341c37d-ce0f-4cd9-9c7b-41d62afabb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:47.816208 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341c37d-ce0f-4cd9-9c7b-41d62afabb5b admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:47.816499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341c37d-ce0f-4cd9-9c7b-41d62afabb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:47.816776 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341c37d-ce0f-4cd9-9c7b-41d62afabb5b admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:47.817456 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c341c37d-ce0f-4cd9-9c7b-41d62afabb5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:47.819805 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c341c37d-ce0f-4cd9-9c7b-41d62afabb5b admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:47.886819 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1101/2210] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:50:47 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:50:47.888562 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224fb13-859d-45ef-a9f6-3b6ab20cf9aa admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:47.902794 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224fb13-859d-45ef-a9f6-3b6ab20cf9aa admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:47.918188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e800a-ee1e-44c6-b332-91007d6ac9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:47.921037 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e800a-ee1e-44c6-b332-91007d6ac9c8 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:47.921037 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e800a-ee1e-44c6-b332-91007d6ac9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:47.921385 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e800a-ee1e-44c6-b332-91007d6ac9c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:47.928296 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e800a-ee1e-44c6-b332-91007d6ac9c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:47.928296 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e800a-ee1e-44c6-b332-91007d6ac9c8 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:47.944881 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1110/2211] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:50:47.946970 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c341c37d-ce0f-4cd9-9c7b-41d62afabb5b admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:47.958190 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c341c37d-ce0f-4cd9-9c7b-41d62afabb5b admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:47.962894 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a4261f-b22e-4917-8a18-865bdc4bc808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:47.963508 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4261f-b22e-4917-8a18-865bdc4bc808 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:47.963949 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4261f-b22e-4917-8a18-865bdc4bc808 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:47.964366 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4261f-b22e-4917-8a18-865bdc4bc808 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:47.965196 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a4261f-b22e-4917-8a18-865bdc4bc808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:47.967933 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a4261f-b22e-4917-8a18-865bdc4bc808 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:47.981354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1102/2212] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 22:50:47.983004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e800a-ee1e-44c6-b332-91007d6ac9c8 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:47.993801 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e800a-ee1e-44c6-b332-91007d6ac9c8 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:47.998983 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad87ed-d73e-4310-889d-501f15ba3e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:47.999933 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad87ed-d73e-4310-889d-501f15ba3e40 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:48.000326 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad87ed-d73e-4310-889d-501f15ba3e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:48.000689 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad87ed-d73e-4310-889d-501f15ba3e40 admin admin] PATH_INFO: `/v3/projects/251bfa4de2b6481db2447261812aebde/users/b6b4a76fd55d462299d20c8842565fb2/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:48.030606 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ad87ed-d73e-4310-889d-501f15ba3e40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=251bfa4de2b6481db2447261812aebde, user_id=b6b4a76fd55d462299d20c8842565fb2, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:48.033944 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ad87ed-d73e-4310-889d-501f15ba3e40 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:48.060718 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1111/2213] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:50:47 2022] PUT /identity/v3/projects/251bfa4de2b6481db2447261812aebde/users/b6b4a76fd55d462299d20c8842565fb2/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:50:48.071513 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd88f7a-cb51-4104-85a8-bf168ad3efd6 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:48.071802 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd88f7a-cb51-4104-85a8-bf168ad3efd6 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:48.072078 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd88f7a-cb51-4104-85a8-bf168ad3efd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:48.105172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-5cd88f7a-cb51-4104-85a8-bf168ad3efd6 None None] MFA Rules not processed for user `b6b4a76fd55d462299d20c8842565fb2`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:50:48.106444 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-5cd88f7a-cb51-4104-85a8-bf168ad3efd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:48.125074 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:50:48.206275 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1103/2214] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:50:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:50:48.218160 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad87ed-d73e-4310-889d-501f15ba3e40 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:48.219874 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-d9ad87ed-d73e-4310-889d-501f15ba3e40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:48.283735 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad87ed-d73e-4310-889d-501f15ba3e40 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:48.287397 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43299b3-d933-4ed5-a7d0-693b25be7038 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:48.288031 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43299b3-d933-4ed5-a7d0-693b25be7038 service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:48.288309 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43299b3-d933-4ed5-a7d0-693b25be7038 service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:48.288580 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43299b3-d933-4ed5-a7d0-693b25be7038 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:48.297190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43299b3-d933-4ed5-a7d0-693b25be7038 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:48.298697 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43299b3-d933-4ed5-a7d0-693b25be7038 service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:48.308884 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1112/2215] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:50:48 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:50:52.014866 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd88f7a-cb51-4104-85a8-bf168ad3efd6 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:52.085656 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd88f7a-cb51-4104-85a8-bf168ad3efd6 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:52.091033 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8736c61-c0e2-4b9c-81bd-026af4f43bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:52.091670 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8736c61-c0e2-4b9c-81bd-026af4f43bd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:52.091949 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8736c61-c0e2-4b9c-81bd-026af4f43bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:52.092218 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8736c61-c0e2-4b9c-81bd-026af4f43bd2 admin admin] PATH_INFO: `/v3/users/b6b4a76fd55d462299d20c8842565fb2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:52.093362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8736c61-c0e2-4b9c-81bd-026af4f43bd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6b4a76fd55d462299d20c8842565fb2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:52.095303 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8736c61-c0e2-4b9c-81bd-026af4f43bd2 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:52.145692 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e8736c61-c0e2-4b9c-81bd-026af4f43bd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6b4a76fd55d462299d20c8842565fb2'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:52.154337 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e8736c61-c0e2-4b9c-81bd-026af4f43bd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6b4a76fd55d462299d20c8842565fb2'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:52.168470 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e8736c61-c0e2-4b9c-81bd-026af4f43bd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6b4a76fd55d462299d20c8842565fb2'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:52.169510 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e8736c61-c0e2-4b9c-81bd-026af4f43bd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6b4a76fd55d462299d20c8842565fb2'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:52.170140 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e8736c61-c0e2-4b9c-81bd-026af4f43bd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6b4a76fd55d462299d20c8842565fb2'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:52.186252 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1104/2216] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:50:52 2022] DELETE /identity/v3/users/b6b4a76fd55d462299d20c8842565fb2 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:50:52.190962 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb96d2-f1fa-44b2-abe8-2dc30a04140d None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:52.191535 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb96d2-f1fa-44b2-abe8-2dc30a04140d None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:52.191535 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb96d2-f1fa-44b2-abe8-2dc30a04140d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:52.211062 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-6aeb96d2-f1fa-44b2-abe8-2dc30a04140d None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:50:52.212318 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-6aeb96d2-f1fa-44b2-abe8-2dc30a04140d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:52.304468 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1113/2217] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:50:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:50:52.312328 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8736c61-c0e2-4b9c-81bd-026af4f43bd2 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:52.313793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-e8736c61-c0e2-4b9c-81bd-026af4f43bd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:52.362505 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:50:52.387003 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8736c61-c0e2-4b9c-81bd-026af4f43bd2 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:52.391305 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad4d0fa-8a5d-4541-a1d7-85176b1b546d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:52.392014 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4d0fa-8a5d-4541-a1d7-85176b1b546d service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:52.392364 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4d0fa-8a5d-4541-a1d7-85176b1b546d service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:52.392741 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4d0fa-8a5d-4541-a1d7-85176b1b546d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:52.403477 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad4d0fa-8a5d-4541-a1d7-85176b1b546d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:52.405018 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad4d0fa-8a5d-4541-a1d7-85176b1b546d service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:52.416533 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1105/2218] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:50:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 22:50:52.698942 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb96d2-f1fa-44b2-abe8-2dc30a04140d None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:52.700408 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-6aeb96d2-f1fa-44b2-abe8-2dc30a04140d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:52.717251 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb96d2-f1fa-44b2-abe8-2dc30a04140d None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:52.727691 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd3b32-2aa6-4787-a3ce-b3a5b64b655c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:52.728342 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd3b32-2aa6-4787-a3ce-b3a5b64b655c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:52.728621 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd3b32-2aa6-4787-a3ce-b3a5b64b655c admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:52.729008 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd3b32-2aa6-4787-a3ce-b3a5b64b655c admin admin] PATH_INFO: `/v3/projects/251bfa4de2b6481db2447261812aebde` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:52.730156 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd3b32-2aa6-4787-a3ce-b3a5b64b655c admin admin] RBAC: Authorizing `identity:delete_project(project_id=251bfa4de2b6481db2447261812aebde)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:52.732094 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd3b32-2aa6-4787-a3ce-b3a5b64b655c admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:52.770760 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:50:52.789772 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-d8dd3b32-2aa6-4787-a3ce-b3a5b64b655c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '251bfa4de2b6481db2447261812aebde', 'request_id': 'req-d8dd3b32-2aa6-4787-a3ce-b3a5b64b655c'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:52.796305 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-d8dd3b32-2aa6-4787-a3ce-b3a5b64b655c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '251bfa4de2b6481db2447261812aebde', 'request_id': 'req-d8dd3b32-2aa6-4787-a3ce-b3a5b64b655c'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:52.806274 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-d8dd3b32-2aa6-4787-a3ce-b3a5b64b655c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '251bfa4de2b6481db2447261812aebde', 'request_id': 'req-d8dd3b32-2aa6-4787-a3ce-b3a5b64b655c'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:52.819763 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-d8dd3b32-2aa6-4787-a3ce-b3a5b64b655c admin admin] The token cache is being invalidate because project 251bfa4de2b6481db2447261812aebde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:50:52.820079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-d8dd3b32-2aa6-4787-a3ce-b3a5b64b655c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:52.820729 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-d8dd3b32-2aa6-4787-a3ce-b3a5b64b655c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:52.822553 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1114/2219] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:50:52 2022] DELETE /identity/v3/projects/251bfa4de2b6481db2447261812aebde => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:50:52.828193 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738146c7-6466-48e7-a7de-7370cac397cc None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:52.828473 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738146c7-6466-48e7-a7de-7370cac397cc None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:52.828741 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738146c7-6466-48e7-a7de-7370cac397cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:52.850694 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-738146c7-6466-48e7-a7de-7370cac397cc None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:50:52.851915 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-738146c7-6466-48e7-a7de-7370cac397cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:52.914985 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1106/2220] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:50:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:50:52.923924 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97e805-feb2-4d0f-bd97-099e40690b2c None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:52.924219 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97e805-feb2-4d0f-bd97-099e40690b2c None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:52.924521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97e805-feb2-4d0f-bd97-099e40690b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:52.954334 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-2d97e805-feb2-4d0f-bd97-099e40690b2c None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:50:52.955977 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-2d97e805-feb2-4d0f-bd97-099e40690b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:52.980218 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1115/2221] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:50:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:50:52.981659 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738146c7-6466-48e7-a7de-7370cac397cc None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:52.996945 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738146c7-6466-48e7-a7de-7370cac397cc None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:53.004812 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a61442-96ab-496b-9a56-df1232a23f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:53.005907 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a61442-96ab-496b-9a56-df1232a23f5f admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:53.008575 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a61442-96ab-496b-9a56-df1232a23f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:53.008575 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a61442-96ab-496b-9a56-df1232a23f5f admin admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:53.008575 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a61442-96ab-496b-9a56-df1232a23f5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:50:53.008575 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a61442-96ab-496b-9a56-df1232a23f5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:53.009677 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a61442-96ab-496b-9a56-df1232a23f5f admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:53.025551 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1107/2222] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:50:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:50:53.030137 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97e805-feb2-4d0f-bd97-099e40690b2c None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:53.046639 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97e805-feb2-4d0f-bd97-099e40690b2c None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:53.064232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2230707-6064-492f-9e7b-6777f7bf3b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:53.064880 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2230707-6064-492f-9e7b-6777f7bf3b3d admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:53.065418 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2230707-6064-492f-9e7b-6777f7bf3b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:53.065575 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2230707-6064-492f-9e7b-6777f7bf3b3d admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:53.066096 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2230707-6064-492f-9e7b-6777f7bf3b3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:53.069236 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2230707-6064-492f-9e7b-6777f7bf3b3d admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:53.110420 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1116/2223] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:50:53 2022] POST /identity/v3/projects => generated 373 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:50:53.115650 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a61442-96ab-496b-9a56-df1232a23f5f admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:53.127457 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a61442-96ab-496b-9a56-df1232a23f5f admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:53.204889 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e63e22-0121-4635-ac08-266513109fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:53.205549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e63e22-0121-4635-ac08-266513109fb9 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:53.206419 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e63e22-0121-4635-ac08-266513109fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:53.206419 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e63e22-0121-4635-ac08-266513109fb9 admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:53.207012 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e63e22-0121-4635-ac08-266513109fb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:53.208987 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e63e22-0121-4635-ac08-266513109fb9 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:53.241334 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1108/2224] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:50:53 2022] POST /identity/v3/users => generated 287 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:50:53.244150 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2230707-6064-492f-9e7b-6777f7bf3b3d admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:53.263708 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2230707-6064-492f-9e7b-6777f7bf3b3d admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:53.269251 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82408029-67c9-4db8-a33a-4d6bc7619375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:53.269895 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82408029-67c9-4db8-a33a-4d6bc7619375 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:53.270173 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82408029-67c9-4db8-a33a-4d6bc7619375 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:53.270440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82408029-67c9-4db8-a33a-4d6bc7619375 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:53.271084 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82408029-67c9-4db8-a33a-4d6bc7619375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:53.272833 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82408029-67c9-4db8-a33a-4d6bc7619375 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:53.284327 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1117/2225] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:50:53.289315 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e63e22-0121-4635-ac08-266513109fb9 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:53.300668 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e63e22-0121-4635-ac08-266513109fb9 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:53.309101 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac686ff2-df1f-440c-9e75-a927b3f0ec6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:53.309845 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac686ff2-df1f-440c-9e75-a927b3f0ec6f admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:53.310209 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac686ff2-df1f-440c-9e75-a927b3f0ec6f admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:53.310797 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac686ff2-df1f-440c-9e75-a927b3f0ec6f admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:53.311623 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac686ff2-df1f-440c-9e75-a927b3f0ec6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:53.313313 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac686ff2-df1f-440c-9e75-a927b3f0ec6f admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:53.329420 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1109/2226] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:50:53.330574 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82408029-67c9-4db8-a33a-4d6bc7619375 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:53.343847 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82408029-67c9-4db8-a33a-4d6bc7619375 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:53.349815 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf47ec-1e7c-4fd5-bad9-99baa253db64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:53.350750 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf47ec-1e7c-4fd5-bad9-99baa253db64 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:53.351204 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf47ec-1e7c-4fd5-bad9-99baa253db64 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:53.351755 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf47ec-1e7c-4fd5-bad9-99baa253db64 admin admin] PATH_INFO: `/v3/projects/2df73c10b6654d47a921aa6866d81526/users/f5fe2bdd1ea54f8cba4d100b26c18464/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:53.375739 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf47ec-1e7c-4fd5-bad9-99baa253db64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2df73c10b6654d47a921aa6866d81526, user_id=f5fe2bdd1ea54f8cba4d100b26c18464, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:53.378621 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf47ec-1e7c-4fd5-bad9-99baa253db64 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:53.397229 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1118/2227] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:50:53 2022] PUT /identity/v3/projects/2df73c10b6654d47a921aa6866d81526/users/f5fe2bdd1ea54f8cba4d100b26c18464/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:50:53.405697 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998ef71-b2ae-470a-ad24-9907e7c481c5 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:53.405986 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998ef71-b2ae-470a-ad24-9907e7c481c5 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:53.406344 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998ef71-b2ae-470a-ad24-9907e7c481c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:53.438966 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-1998ef71-b2ae-470a-ad24-9907e7c481c5 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:50:53.440201 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-1998ef71-b2ae-470a-ad24-9907e7c481c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:53.491619 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:50:53.514509 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1110/2228] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:50:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:50:53.524369 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf47ec-1e7c-4fd5-bad9-99baa253db64 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:53.526178 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-20bf47ec-1e7c-4fd5-bad9-99baa253db64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:53.591741 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf47ec-1e7c-4fd5-bad9-99baa253db64 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:53.595512 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b0f9d-c158-498f-a537-776fc0a6667c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:53.596122 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b0f9d-c158-498f-a537-776fc0a6667c service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:53.596407 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b0f9d-c158-498f-a537-776fc0a6667c service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:53.596682 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b0f9d-c158-498f-a537-776fc0a6667c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:53.607023 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b0f9d-c158-498f-a537-776fc0a6667c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:53.608497 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b0f9d-c158-498f-a537-776fc0a6667c service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:53.618615 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1119/2229] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:50:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:50:56.087666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1998ef71-b2ae-470a-ad24-9907e7c481c5 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:56.089939 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-1998ef71-b2ae-470a-ad24-9907e7c481c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:56.102720 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1998ef71-b2ae-470a-ad24-9907e7c481c5 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:56.108613 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59b150-f6a5-4d8f-9098-1b9c90c8d4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:56.109415 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59b150-f6a5-4d8f-9098-1b9c90c8d4a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:56.109698 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59b150-f6a5-4d8f-9098-1b9c90c8d4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:56.109971 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59b150-f6a5-4d8f-9098-1b9c90c8d4a7 admin admin] PATH_INFO: `/v3/users/84b8d41c742d40499ef0a719e1211097` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:56.111149 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a59b150-f6a5-4d8f-9098-1b9c90c8d4a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84b8d41c742d40499ef0a719e1211097)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:56.113092 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a59b150-f6a5-4d8f-9098-1b9c90c8d4a7 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:56.188015 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-5a59b150-f6a5-4d8f-9098-1b9c90c8d4a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84b8d41c742d40499ef0a719e1211097'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:56.196951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-5a59b150-f6a5-4d8f-9098-1b9c90c8d4a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84b8d41c742d40499ef0a719e1211097'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:56.209847 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-5a59b150-f6a5-4d8f-9098-1b9c90c8d4a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84b8d41c742d40499ef0a719e1211097'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:56.210584 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-5a59b150-f6a5-4d8f-9098-1b9c90c8d4a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84b8d41c742d40499ef0a719e1211097'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:56.211196 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-5a59b150-f6a5-4d8f-9098-1b9c90c8d4a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84b8d41c742d40499ef0a719e1211097'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:56.227508 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1111/2230] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:50:56 2022] DELETE /identity/v3/users/84b8d41c742d40499ef0a719e1211097 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:50:56.232761 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00f5b7-20ed-4879-b383-fedbe487ee3a None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:56.233228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00f5b7-20ed-4879-b383-fedbe487ee3a None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:56.233884 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00f5b7-20ed-4879-b383-fedbe487ee3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:56.259901 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-ac00f5b7-20ed-4879-b383-fedbe487ee3a None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:50:56.261976 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-ac00f5b7-20ed-4879-b383-fedbe487ee3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:56.324256 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:50:56.345658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1120/2231] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:50:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:50:56.358328 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59b150-f6a5-4d8f-9098-1b9c90c8d4a7 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:56.359900 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-5a59b150-f6a5-4d8f-9098-1b9c90c8d4a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:56.429381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59b150-f6a5-4d8f-9098-1b9c90c8d4a7 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:56.434643 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca8077-1e5d-4946-87c7-d461ad66dcc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:56.435126 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca8077-1e5d-4946-87c7-d461ad66dcc1 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:56.435410 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca8077-1e5d-4946-87c7-d461ad66dcc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:56.435682 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca8077-1e5d-4946-87c7-d461ad66dcc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:56.452816 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca8077-1e5d-4946-87c7-d461ad66dcc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:56.457560 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca8077-1e5d-4946-87c7-d461ad66dcc1 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:56.469583 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1112/2232] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:50:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:50:57.160317 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac00f5b7-20ed-4879-b383-fedbe487ee3a None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:57.161988 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-ac00f5b7-20ed-4879-b383-fedbe487ee3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:57.175464 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac00f5b7-20ed-4879-b383-fedbe487ee3a None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:57.180459 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64075824-c14f-4ad2-92cf-d3e9ae6f7f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:57.189372 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64075824-c14f-4ad2-92cf-d3e9ae6f7f3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:57.189930 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64075824-c14f-4ad2-92cf-d3e9ae6f7f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:57.190211 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64075824-c14f-4ad2-92cf-d3e9ae6f7f3f admin admin] PATH_INFO: `/v3/projects/6e9d10d60f3b4918a3dc554d93c86d7b` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:57.193210 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64075824-c14f-4ad2-92cf-d3e9ae6f7f3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e9d10d60f3b4918a3dc554d93c86d7b)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:57.196052 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64075824-c14f-4ad2-92cf-d3e9ae6f7f3f admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:57.273342 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-64075824-c14f-4ad2-92cf-d3e9ae6f7f3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e9d10d60f3b4918a3dc554d93c86d7b', 'request_id': 'req-64075824-c14f-4ad2-92cf-d3e9ae6f7f3f'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:57.283543 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-64075824-c14f-4ad2-92cf-d3e9ae6f7f3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e9d10d60f3b4918a3dc554d93c86d7b', 'request_id': 'req-64075824-c14f-4ad2-92cf-d3e9ae6f7f3f'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:57.295862 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-64075824-c14f-4ad2-92cf-d3e9ae6f7f3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e9d10d60f3b4918a3dc554d93c86d7b', 'request_id': 'req-64075824-c14f-4ad2-92cf-d3e9ae6f7f3f'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:57.310039 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-64075824-c14f-4ad2-92cf-d3e9ae6f7f3f admin admin] The token cache is being invalidate because project 6e9d10d60f3b4918a3dc554d93c86d7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:50:57.310669 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-64075824-c14f-4ad2-92cf-d3e9ae6f7f3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:57.311546 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-64075824-c14f-4ad2-92cf-d3e9ae6f7f3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:50:57.314192 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1121/2233] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:50:57 2022] DELETE /identity/v3/projects/6e9d10d60f3b4918a3dc554d93c86d7b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:50:57.319465 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ee893-f299-4926-8c59-ed664e193fc5 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:57.320379 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ee893-f299-4926-8c59-ed664e193fc5 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:57.321325 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ee893-f299-4926-8c59-ed664e193fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:57.349883 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-433ee893-f299-4926-8c59-ed664e193fc5 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:50:57.351403 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-433ee893-f299-4926-8c59-ed664e193fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:57.445613 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:50:57.469317 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1113/2234] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:50:57.527325 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537f792-d937-4fb0-9c6f-a787525b194c None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:57.528235 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537f792-d937-4fb0-9c6f-a787525b194c None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:57.528516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537f792-d937-4fb0-9c6f-a787525b194c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:57.551528 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-e537f792-d937-4fb0-9c6f-a787525b194c None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:50:57.552978 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-e537f792-d937-4fb0-9c6f-a787525b194c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:57.561816 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1122/2235] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:50:57.566181 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ee893-f299-4926-8c59-ed664e193fc5 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:57.577918 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ee893-f299-4926-8c59-ed664e193fc5 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:57.583014 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d00af-cf56-4db5-9d55-cfa2b916c6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:57.584100 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d00af-cf56-4db5-9d55-cfa2b916c6ae admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:57.584100 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d00af-cf56-4db5-9d55-cfa2b916c6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:57.584296 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d00af-cf56-4db5-9d55-cfa2b916c6ae admin admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:57.585041 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d00af-cf56-4db5-9d55-cfa2b916c6ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:50:57.585449 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d00af-cf56-4db5-9d55-cfa2b916c6ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:57.587208 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d00af-cf56-4db5-9d55-cfa2b916c6ae admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:57.601616 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1114/2236] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:50:57.606326 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e537f792-d937-4fb0-9c6f-a787525b194c None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:57.616084 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e537f792-d937-4fb0-9c6f-a787525b194c None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:57.622175 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df35db-fe1e-43e5-bf28-6517955391fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:57.622810 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df35db-fe1e-43e5-bf28-6517955391fa admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:57.623090 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df35db-fe1e-43e5-bf28-6517955391fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:57.623359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df35db-fe1e-43e5-bf28-6517955391fa admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:57.623995 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df35db-fe1e-43e5-bf28-6517955391fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:57.626569 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df35db-fe1e-43e5-bf28-6517955391fa admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:57.658511 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1123/2237] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:50:57 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:50:57.662985 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d00af-cf56-4db5-9d55-cfa2b916c6ae admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:57.675501 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d00af-cf56-4db5-9d55-cfa2b916c6ae admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:57.760545 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b9b8f-e91d-4136-aa26-baf5436c6c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:57.761884 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b9b8f-e91d-4136-aa26-baf5436c6c80 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:57.762608 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b9b8f-e91d-4136-aa26-baf5436c6c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:57.763211 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b9b8f-e91d-4136-aa26-baf5436c6c80 admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:57.764441 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2b9b8f-e91d-4136-aa26-baf5436c6c80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:57.768117 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2b9b8f-e91d-4136-aa26-baf5436c6c80 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:57.794453 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1115/2238] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:50:57 2022] POST /identity/v3/users => generated 287 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:50:57.798588 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df35db-fe1e-43e5-bf28-6517955391fa admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:57.815999 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df35db-fe1e-43e5-bf28-6517955391fa admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:57.821473 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e89c449-dbff-4943-9fad-77fc8bc70fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:57.822342 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89c449-dbff-4943-9fad-77fc8bc70fe2 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:57.823139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89c449-dbff-4943-9fad-77fc8bc70fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:57.823526 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89c449-dbff-4943-9fad-77fc8bc70fe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:57.824039 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e89c449-dbff-4943-9fad-77fc8bc70fe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:57.825836 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e89c449-dbff-4943-9fad-77fc8bc70fe2 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:57.970383 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1124/2239] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:50:57 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:50:57.974690 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b9b8f-e91d-4136-aa26-baf5436c6c80 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:57.986659 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b9b8f-e91d-4136-aa26-baf5436c6c80 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:57.995313 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3978351f-76a2-4376-a776-c3dae5cbf2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:57.998034 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978351f-76a2-4376-a776-c3dae5cbf2d6 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:57.999190 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978351f-76a2-4376-a776-c3dae5cbf2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:57.999568 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978351f-76a2-4376-a776-c3dae5cbf2d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:58.000387 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3978351f-76a2-4376-a776-c3dae5cbf2d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:58.002333 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3978351f-76a2-4376-a776-c3dae5cbf2d6 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:58.021775 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1116/2240] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:50:57 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:50:58.023061 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e89c449-dbff-4943-9fad-77fc8bc70fe2 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:58.034306 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e89c449-dbff-4943-9fad-77fc8bc70fe2 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:58.039750 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b6e69-882c-4c60-b060-2fd999d3bdd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:58.042932 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b6e69-882c-4c60-b060-2fd999d3bdd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:58.043339 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b6e69-882c-4c60-b060-2fd999d3bdd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:58.043847 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b6e69-882c-4c60-b060-2fd999d3bdd9 admin admin] PATH_INFO: `/v3/projects/0fb67f0ac33e4d59adca72eaabce5592/users/d0a0b95fe55749afb87be40f193942fe/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:58.061269 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b6e69-882c-4c60-b060-2fd999d3bdd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fb67f0ac33e4d59adca72eaabce5592, user_id=d0a0b95fe55749afb87be40f193942fe, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:58.064504 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b6e69-882c-4c60-b060-2fd999d3bdd9 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:58.093258 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1125/2241] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:50:58 2022] PUT /identity/v3/projects/0fb67f0ac33e4d59adca72eaabce5592/users/d0a0b95fe55749afb87be40f193942fe/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:50:58.095235 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b74e8-3465-4852-8e0c-26cc08fa0f65 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:58.095524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b74e8-3465-4852-8e0c-26cc08fa0f65 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:58.095799 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b74e8-3465-4852-8e0c-26cc08fa0f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:58.131721 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-eb7b74e8-3465-4852-8e0c-26cc08fa0f65 None None] MFA Rules not processed for user `d0a0b95fe55749afb87be40f193942fe`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:50:58.133028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-eb7b74e8-3465-4852-8e0c-26cc08fa0f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:58.261505 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1117/2242] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:50:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:50:58.274002 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b6e69-882c-4c60-b060-2fd999d3bdd9 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:58.278098 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-9d8b6e69-882c-4c60-b060-2fd999d3bdd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:58.375452 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b6e69-882c-4c60-b060-2fd999d3bdd9 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:58.379926 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176f6d0-5a2c-4900-afdb-dcd8ed3a767e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:58.381220 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176f6d0-5a2c-4900-afdb-dcd8ed3a767e service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:58.381545 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176f6d0-5a2c-4900-afdb-dcd8ed3a767e service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:58.381813 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176f6d0-5a2c-4900-afdb-dcd8ed3a767e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:58.390673 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176f6d0-5a2c-4900-afdb-dcd8ed3a767e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:58.392098 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176f6d0-5a2c-4900-afdb-dcd8ed3a767e service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:58.402623 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1126/2243] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:50:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:50:59.129770 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6da504-a047-48d4-9fc3-fa5eeb00cfad None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:59.129770 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6da504-a047-48d4-9fc3-fa5eeb00cfad None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:59.129770 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6da504-a047-48d4-9fc3-fa5eeb00cfad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:59.203541 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-3c6da504-a047-48d4-9fc3-fa5eeb00cfad None None] MFA Rules not processed for user `f5fe2bdd1ea54f8cba4d100b26c18464`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:50:59.205598 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-3c6da504-a047-48d4-9fc3-fa5eeb00cfad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:59.270073 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:50:59.310175 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1118/2244] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:50:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:50:59.315817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176f6d0-5a2c-4900-afdb-dcd8ed3a767e service cinder] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:59.317635 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-7176f6d0-5a2c-4900-afdb-dcd8ed3a767e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:50:59.343096 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:50:59.426083 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176f6d0-5a2c-4900-afdb-dcd8ed3a767e service cinder] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:59.434488 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a9dff0-bab7-40e4-b903-2961f2db606b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:59.435377 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9dff0-bab7-40e4-b903-2961f2db606b admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:59.435826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9dff0-bab7-40e4-b903-2961f2db606b admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:59.436246 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9dff0-bab7-40e4-b903-2961f2db606b admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:59.437734 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a9dff0-bab7-40e4-b903-2961f2db606b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:59.442221 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a9dff0-bab7-40e4-b903-2961f2db606b admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:59.490089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1127/2245] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:50:59 2022] POST /identity/v3/projects => generated 373 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:50:59.496480 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6da504-a047-48d4-9fc3-fa5eeb00cfad None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:59.519584 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6da504-a047-48d4-9fc3-fa5eeb00cfad None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:59.643128 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceeff0f-f4fc-4ee1-9d92-5bc3bdea3801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:59.644361 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceeff0f-f4fc-4ee1-9d92-5bc3bdea3801 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:59.644719 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceeff0f-f4fc-4ee1-9d92-5bc3bdea3801 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:59.645531 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceeff0f-f4fc-4ee1-9d92-5bc3bdea3801 admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:59.646619 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceeff0f-f4fc-4ee1-9d92-5bc3bdea3801 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:59.650919 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceeff0f-f4fc-4ee1-9d92-5bc3bdea3801 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:59.709026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1119/2246] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:50:59 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:50:59.710495 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a9dff0-bab7-40e4-b903-2961f2db606b admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:59.732175 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a9dff0-bab7-40e4-b903-2961f2db606b admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:59.739058 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f975ee-5562-451c-8b04-5a97b4223734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:59.739725 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f975ee-5562-451c-8b04-5a97b4223734 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:59.740002 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f975ee-5562-451c-8b04-5a97b4223734 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:59.741201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f975ee-5562-451c-8b04-5a97b4223734 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:59.741908 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f975ee-5562-451c-8b04-5a97b4223734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:59.744428 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f975ee-5562-451c-8b04-5a97b4223734 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:59.761333 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1128/2247] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:50:59.764016 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceeff0f-f4fc-4ee1-9d92-5bc3bdea3801 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:59.782659 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceeff0f-f4fc-4ee1-9d92-5bc3bdea3801 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:59.796025 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3eb22-46f5-476a-ad97-7048b0a3c5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:59.797171 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3eb22-46f5-476a-ad97-7048b0a3c5f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:59.797743 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3eb22-46f5-476a-ad97-7048b0a3c5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:59.800069 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3eb22-46f5-476a-ad97-7048b0a3c5f1 admin admin] PATH_INFO: `/v3/projects/bc5da3cccdac4bb8b3a7ebb989d431b9/users/1df55d4c35794a33a6133c19412da7c6/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:59.820252 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea3eb22-46f5-476a-ad97-7048b0a3c5f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc5da3cccdac4bb8b3a7ebb989d431b9, user_id=1df55d4c35794a33a6133c19412da7c6, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:59.823584 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea3eb22-46f5-476a-ad97-7048b0a3c5f1 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:59.843537 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1120/2248] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:50:59 2022] PUT /identity/v3/projects/bc5da3cccdac4bb8b3a7ebb989d431b9/users/1df55d4c35794a33a6133c19412da7c6/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:50:59.848935 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f975ee-5562-451c-8b04-5a97b4223734 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:50:59.870110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f975ee-5562-451c-8b04-5a97b4223734 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:50:59.972145 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea5d30-515f-4f6e-9bd9-e519bd76e4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:50:59.973194 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea5d30-515f-4f6e-9bd9-e519bd76e4de admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:50:59.973611 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea5d30-515f-4f6e-9bd9-e519bd76e4de admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:50:59.974018 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea5d30-515f-4f6e-9bd9-e519bd76e4de admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:50:59.977521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcea5d30-515f-4f6e-9bd9-e519bd76e4de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:50:59.977521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcea5d30-515f-4f6e-9bd9-e519bd76e4de admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:50:59.990210 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1129/2249] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:50:59.995211 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3eb22-46f5-476a-ad97-7048b0a3c5f1 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:00.006309 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3eb22-46f5-476a-ad97-7048b0a3c5f1 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:00.012901 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa6036-7daf-4fdf-913a-f1ef3e3a858e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:00.013800 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa6036-7daf-4fdf-913a-f1ef3e3a858e admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:00.013941 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa6036-7daf-4fdf-913a-f1ef3e3a858e admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:00.014190 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa6036-7daf-4fdf-913a-f1ef3e3a858e admin admin] PATH_INFO: `/v3/projects/bc5da3cccdac4bb8b3a7ebb989d431b9/users/1df55d4c35794a33a6133c19412da7c6/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:00.016237 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa6036-7daf-4fdf-913a-f1ef3e3a858e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc5da3cccdac4bb8b3a7ebb989d431b9, user_id=1df55d4c35794a33a6133c19412da7c6, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:00.018991 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa6036-7daf-4fdf-913a-f1ef3e3a858e admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:00.031704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1121/2250] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:50:59 2022] PUT /identity/v3/projects/bc5da3cccdac4bb8b3a7ebb989d431b9/users/1df55d4c35794a33a6133c19412da7c6/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:51:01.797229 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea5d30-515f-4f6e-9bd9-e519bd76e4de admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:01.815801 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea5d30-515f-4f6e-9bd9-e519bd76e4de admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:01.953042 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101741d6-5be0-4942-9cc0-8089ecfafd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:01.953743 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101741d6-5be0-4942-9cc0-8089ecfafd4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:01.954020 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101741d6-5be0-4942-9cc0-8089ecfafd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:01.954286 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101741d6-5be0-4942-9cc0-8089ecfafd4d admin admin] PATH_INFO: `/v3/users/d0a0b95fe55749afb87be40f193942fe` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:01.955530 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101741d6-5be0-4942-9cc0-8089ecfafd4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0a0b95fe55749afb87be40f193942fe)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:01.957575 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101741d6-5be0-4942-9cc0-8089ecfafd4d admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:02.028678 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:02.050982 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-101741d6-5be0-4942-9cc0-8089ecfafd4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0a0b95fe55749afb87be40f193942fe'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:51:02.075747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-101741d6-5be0-4942-9cc0-8089ecfafd4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0a0b95fe55749afb87be40f193942fe'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:51:02.093611 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-101741d6-5be0-4942-9cc0-8089ecfafd4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0a0b95fe55749afb87be40f193942fe'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:51:02.095293 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-101741d6-5be0-4942-9cc0-8089ecfafd4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0a0b95fe55749afb87be40f193942fe'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:51:02.095828 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-101741d6-5be0-4942-9cc0-8089ecfafd4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0a0b95fe55749afb87be40f193942fe'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:51:02.138838 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1130/2251] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:51:01 2022] DELETE /identity/v3/users/d0a0b95fe55749afb87be40f193942fe => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:51:02.143863 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4722d-6cbd-4f44-870c-3355902e8b5d None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:02.144312 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4722d-6cbd-4f44-870c-3355902e8b5d None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:02.144393 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4722d-6cbd-4f44-870c-3355902e8b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:02.188105 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-90e4722d-6cbd-4f44-870c-3355902e8b5d None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:51:02.195986 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-90e4722d-6cbd-4f44-870c-3355902e8b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:02.310497 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1122/2252] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:51:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:51:02.333432 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101741d6-5be0-4942-9cc0-8089ecfafd4d admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:02.334890 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-101741d6-5be0-4942-9cc0-8089ecfafd4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:02.492118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101741d6-5be0-4942-9cc0-8089ecfafd4d admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:02.498979 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87845b96-6547-42e8-a9b5-f062a76ffbef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:02.499611 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87845b96-6547-42e8-a9b5-f062a76ffbef service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:02.499885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87845b96-6547-42e8-a9b5-f062a76ffbef service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:02.502252 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87845b96-6547-42e8-a9b5-f062a76ffbef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:02.542559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87845b96-6547-42e8-a9b5-f062a76ffbef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:02.544288 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87845b96-6547-42e8-a9b5-f062a76ffbef service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:02.560905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1131/2253] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:51:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:51:03.107132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e4722d-6cbd-4f44-870c-3355902e8b5d None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:03.108636 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-90e4722d-6cbd-4f44-870c-3355902e8b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:03.132196 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e4722d-6cbd-4f44-870c-3355902e8b5d None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:03.138955 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e440a2a8-691c-4fdd-b449-9db6d69744d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:03.139799 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440a2a8-691c-4fdd-b449-9db6d69744d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:03.140119 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440a2a8-691c-4fdd-b449-9db6d69744d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:03.140397 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440a2a8-691c-4fdd-b449-9db6d69744d4 admin admin] PATH_INFO: `/v3/projects/0fb67f0ac33e4d59adca72eaabce5592` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:03.143011 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e440a2a8-691c-4fdd-b449-9db6d69744d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fb67f0ac33e4d59adca72eaabce5592)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:03.144971 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e440a2a8-691c-4fdd-b449-9db6d69744d4 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:03.212327 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:03.235485 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e440a2a8-691c-4fdd-b449-9db6d69744d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fb67f0ac33e4d59adca72eaabce5592', 'request_id': 'req-e440a2a8-691c-4fdd-b449-9db6d69744d4'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:51:03.248590 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e440a2a8-691c-4fdd-b449-9db6d69744d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fb67f0ac33e4d59adca72eaabce5592', 'request_id': 'req-e440a2a8-691c-4fdd-b449-9db6d69744d4'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:51:03.255397 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e440a2a8-691c-4fdd-b449-9db6d69744d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fb67f0ac33e4d59adca72eaabce5592', 'request_id': 'req-e440a2a8-691c-4fdd-b449-9db6d69744d4'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:51:03.268808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e440a2a8-691c-4fdd-b449-9db6d69744d4 admin admin] The token cache is being invalidate because project 0fb67f0ac33e4d59adca72eaabce5592 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:51:03.269091 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e440a2a8-691c-4fdd-b449-9db6d69744d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:51:03.270090 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e440a2a8-691c-4fdd-b449-9db6d69744d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:51:03.272347 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1123/2254] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:51:03 2022] DELETE /identity/v3/projects/0fb67f0ac33e4d59adca72eaabce5592 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:51:05.702101 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86066f41-c2e5-4fdc-9375-a77eaa04965b None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:05.702609 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86066f41-c2e5-4fdc-9375-a77eaa04965b None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:05.702692 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86066f41-c2e5-4fdc-9375-a77eaa04965b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:05.735399 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-86066f41-c2e5-4fdc-9375-a77eaa04965b None None] MFA Rules not processed for user `1df55d4c35794a33a6133c19412da7c6`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:51:05.736632 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-86066f41-c2e5-4fdc-9375-a77eaa04965b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:05.836051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1132/2255] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:51:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:51:05.850022 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e440a2a8-691c-4fdd-b449-9db6d69744d4 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:05.851429 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-e440a2a8-691c-4fdd-b449-9db6d69744d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:05.939680 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e440a2a8-691c-4fdd-b449-9db6d69744d4 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:05.945396 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499cb7ac-fc93-4ecd-a348-47f4be0dec9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:05.946018 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cb7ac-fc93-4ecd-a348-47f4be0dec9b service nova] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:05.946299 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cb7ac-fc93-4ecd-a348-47f4be0dec9b service nova] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:05.946576 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cb7ac-fc93-4ecd-a348-47f4be0dec9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:05.948331 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-499cb7ac-fc93-4ecd-a348-47f4be0dec9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:06.010176 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499cb7ac-fc93-4ecd-a348-47f4be0dec9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:06.011786 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499cb7ac-fc93-4ecd-a348-47f4be0dec9b service nova] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:06.021623 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1124/2256] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:51:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:51:07.400466 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d89229-d576-4635-85c3-e8123606a325 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:07.400945 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d89229-d576-4635-85c3-e8123606a325 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:07.401041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d89229-d576-4635-85c3-e8123606a325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:07.421737 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-f7d89229-d576-4635-85c3-e8123606a325 None None] MFA Rules not processed for user `f5fe2bdd1ea54f8cba4d100b26c18464`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:51:07.423205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-f7d89229-d576-4635-85c3-e8123606a325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:07.431663 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1133/2257] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:51:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:51:16.017388 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=69,keystone:DELETE=5,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:17.421234 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=81,keystone:DELETE=5,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:26.206009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66eb581-b0ca-4a11-a30a-8a829a24c41b None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:26.206600 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66eb581-b0ca-4a11-a30a-8a829a24c41b None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:26.206600 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66eb581-b0ca-4a11-a30a-8a829a24c41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:26.225896 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-f66eb581-b0ca-4a11-a30a-8a829a24c41b None None] MFA Rules not processed for user `0f3c2f531a5942e6afabb16521523754`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:51:26.227126 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-f66eb581-b0ca-4a11-a30a-8a829a24c41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:26.282221 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1125/2258] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:51:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:51:36.275596 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=25 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:47.688567 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d89229-d576-4635-85c3-e8123606a325 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:51:47.690095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-f7d89229-d576-4635-85c3-e8123606a325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:47.753470 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d89229-d576-4635-85c3-e8123606a325 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:51:47.757514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ed719-86bc-419c-88f4-9ca282cccd69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:51:47.758310 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ed719-86bc-419c-88f4-9ca282cccd69 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:47.758663 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ed719-86bc-419c-88f4-9ca282cccd69 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:47.759028 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ed719-86bc-419c-88f4-9ca282cccd69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:47.760878 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-344ed719-86bc-419c-88f4-9ca282cccd69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:47.828938 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ed719-86bc-419c-88f4-9ca282cccd69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:51:47.830456 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ed719-86bc-419c-88f4-9ca282cccd69 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:51:47.841327 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1134/2259] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:51:47 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:51:49.172371 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f4bd7-006a-40a5-8658-b5d2786ac80d None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:51:49.172933 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f4bd7-006a-40a5-8658-b5d2786ac80d None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:51:49.172933 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f4bd7-006a-40a5-8658-b5d2786ac80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:51:49.192662 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-bb9f4bd7-006a-40a5-8658-b5d2786ac80d None None] MFA Rules not processed for user `6367371496f04cc2a7c105fbbed68dca`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:51:49.193929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-bb9f4bd7-006a-40a5-8658-b5d2786ac80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:51:49.257596 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1126/2260] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:51:49 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:51:57.836860 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=50 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:51:59.248410 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=25 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:22.680117 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ed719-86bc-419c-88f4-9ca282cccd69 service neutron] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:22.682184 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-344ed719-86bc-419c-88f4-9ca282cccd69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:22.764039 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ed719-86bc-419c-88f4-9ca282cccd69 service neutron] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:22.770461 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da063e6e-aa98-4ce4-8d70-5c9a1b8c4e31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:22.771512 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da063e6e-aa98-4ce4-8d70-5c9a1b8c4e31 service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:22.771795 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da063e6e-aa98-4ce4-8d70-5c9a1b8c4e31 service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:22.772154 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da063e6e-aa98-4ce4-8d70-5c9a1b8c4e31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:22.774903 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-da063e6e-aa98-4ce4-8d70-5c9a1b8c4e31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:22.795644 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da063e6e-aa98-4ce4-8d70-5c9a1b8c4e31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:22.797548 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da063e6e-aa98-4ce4-8d70-5c9a1b8c4e31 service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:22.817728 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1135/2261] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:52:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:52:23.498409 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a36161-1327-4b63-aa9f-293ac455957f None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:23.499255 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a36161-1327-4b63-aa9f-293ac455957f None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:23.500222 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a36161-1327-4b63-aa9f-293ac455957f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:23.526390 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-04a36161-1327-4b63-aa9f-293ac455957f None None] MFA Rules not processed for user `da22fc66a1d14fa983463331de110258`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:23.532358 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-04a36161-1327-4b63-aa9f-293ac455957f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:23.602654 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1127/2262] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:52:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:52:25.477026 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da063e6e-aa98-4ce4-8d70-5c9a1b8c4e31 service cinder] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:25.478528 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-da063e6e-aa98-4ce4-8d70-5c9a1b8c4e31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:25.491703 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da063e6e-aa98-4ce4-8d70-5c9a1b8c4e31 service cinder] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:25.495645 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e1201-85c4-4aca-8ff9-4324f3e8f291 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:25.496252 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e1201-85c4-4aca-8ff9-4324f3e8f291 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:25.496533 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e1201-85c4-4aca-8ff9-4324f3e8f291 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:25.496805 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e1201-85c4-4aca-8ff9-4324f3e8f291 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:25.498604 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-956e1201-85c4-4aca-8ff9-4324f3e8f291 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:25.513558 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e1201-85c4-4aca-8ff9-4324f3e8f291 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:25.515151 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e1201-85c4-4aca-8ff9-4324f3e8f291 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:25.527368 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1136/2263] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:52:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:52:26.043012 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a36161-1327-4b63-aa9f-293ac455957f None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:26.053912 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a36161-1327-4b63-aa9f-293ac455957f None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:26.057814 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ccfc7-32a2-483a-97bd-cdaa453d5d9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:26.058504 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ccfc7-32a2-483a-97bd-cdaa453d5d9f service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:26.058864 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ccfc7-32a2-483a-97bd-cdaa453d5d9f service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:26.059207 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ccfc7-32a2-483a-97bd-cdaa453d5d9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:26.069136 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ccfc7-32a2-483a-97bd-cdaa453d5d9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:26.070621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ccfc7-32a2-483a-97bd-cdaa453d5d9f service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:26.082786 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1128/2264] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:52:26 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:52:30.451712 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611b1af-0c48-4199-93ef-9939bfe60629 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:30.452161 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611b1af-0c48-4199-93ef-9939bfe60629 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:30.452253 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611b1af-0c48-4199-93ef-9939bfe60629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:30.473527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-9611b1af-0c48-4199-93ef-9939bfe60629 None None] MFA Rules not processed for user `aff5e804c64345b298c745287a1d0a32`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:30.475000 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-9611b1af-0c48-4199-93ef-9939bfe60629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:30.542098 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1137/2265] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 10 22:52:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:52:30.632325 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ccfc7-32a2-483a-97bd-cdaa453d5d9f service neutron] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:30.633889 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-9b8ccfc7-32a2-483a-97bd-cdaa453d5d9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:30.721925 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ccfc7-32a2-483a-97bd-cdaa453d5d9f service neutron] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:30.728697 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf5580-3fa6-4e09-81bc-8a2f7ce60b6c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e3a76df95a34b9fb9816f22b85d1071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:30.729270 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf5580-3fa6-4e09-81bc-8a2f7ce60b6c service swift] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:30.729570 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf5580-3fa6-4e09-81bc-8a2f7ce60b6c service swift] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:30.729839 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf5580-3fa6-4e09-81bc-8a2f7ce60b6c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:30.740123 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcf5580-3fa6-4e09-81bc-8a2f7ce60b6c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:30.742279 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcf5580-3fa6-4e09-81bc-8a2f7ce60b6c service swift] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:30.751612 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1129/2266] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 10 22:52:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 22:52:33.529017 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9611b1af-0c48-4199-93ef-9939bfe60629 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:33.530523 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-9611b1af-0c48-4199-93ef-9939bfe60629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:33.548453 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9611b1af-0c48-4199-93ef-9939bfe60629 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:33.555079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ab983-2a19-4581-b968-46313532a351 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:33.555694 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ab983-2a19-4581-b968-46313532a351 service nova] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:33.555973 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ab983-2a19-4581-b968-46313532a351 service nova] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:33.556239 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ab983-2a19-4581-b968-46313532a351 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:33.558135 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-885ab983-2a19-4581-b968-46313532a351 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:33.636351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885ab983-2a19-4581-b968-46313532a351 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:33.638074 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885ab983-2a19-4581-b968-46313532a351 service nova] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:33.655223 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1138/2267] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:52:33 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:52:40.749236 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=63 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:42.119149 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf5580-3fa6-4e09-81bc-8a2f7ce60b6c service swift] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:42.132221 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf5580-3fa6-4e09-81bc-8a2f7ce60b6c service swift] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:42.144548 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81af17-3388-42a3-85a4-9b280dba75df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:42.145384 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81af17-3388-42a3-85a4-9b280dba75df service nova] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:42.145673 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81af17-3388-42a3-85a4-9b280dba75df service nova] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:42.145948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81af17-3388-42a3-85a4-9b280dba75df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:42.147946 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-ad81af17-3388-42a3-85a4-9b280dba75df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:42.160709 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81af17-3388-42a3-85a4-9b280dba75df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:42.162412 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81af17-3388-42a3-85a4-9b280dba75df service nova] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:42.176129 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1130/2268] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:52:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:52:43.645811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=85 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:43.957040 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ab983-2a19-4581-b968-46313532a351 service nova] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:43.968974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ab983-2a19-4581-b968-46313532a351 service nova] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:43.975138 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdc13a-d3d2-4686-a372-1df2167d4c03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:43.975758 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdc13a-d3d2-4686-a372-1df2167d4c03 service nova] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:43.976038 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdc13a-d3d2-4686-a372-1df2167d4c03 service nova] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:43.976349 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdc13a-d3d2-4686-a372-1df2167d4c03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:43.978174 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-e0bdc13a-d3d2-4686-a372-1df2167d4c03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:44.043987 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bdc13a-d3d2-4686-a372-1df2167d4c03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:44.045624 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bdc13a-d3d2-4686-a372-1df2167d4c03 service nova] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:44.055583 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1139/2269] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:52:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:52:52.170951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=6 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:52:52.392034 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81af17-3388-42a3-85a4-9b280dba75df service nova] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:52.402295 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81af17-3388-42a3-85a4-9b280dba75df service nova] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:52.407518 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f509c30-fc00-483e-a2d5-8031829e67f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:52.408163 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f509c30-fc00-483e-a2d5-8031829e67f5 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:52.408440 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f509c30-fc00-483e-a2d5-8031829e67f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:52.408720 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f509c30-fc00-483e-a2d5-8031829e67f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:52.411430 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-2f509c30-fc00-483e-a2d5-8031829e67f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:52.423092 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f509c30-fc00-483e-a2d5-8031829e67f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:52.424552 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f509c30-fc00-483e-a2d5-8031829e67f5 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:52.434595 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1131/2270] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:52:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:52:52.628068 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdc13a-d3d2-4686-a372-1df2167d4c03 service nova] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:52.644247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdc13a-d3d2-4686-a372-1df2167d4c03 service nova] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:52.651958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d8279d-e522-4de8-9c6a-5204f59dbe72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:52.653348 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8279d-e522-4de8-9c6a-5204f59dbe72 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:52.653996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8279d-e522-4de8-9c6a-5204f59dbe72 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:52.654353 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8279d-e522-4de8-9c6a-5204f59dbe72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:52.656705 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-a4d8279d-e522-4de8-9c6a-5204f59dbe72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:52.669698 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d8279d-e522-4de8-9c6a-5204f59dbe72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:52.671662 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d8279d-e522-4de8-9c6a-5204f59dbe72 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:52.680305 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1140/2271] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:52:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:52:52.845077 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b7691-a3de-4720-a872-03a1b0b6a8f2 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:52.845491 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b7691-a3de-4720-a872-03a1b0b6a8f2 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:52.845948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b7691-a3de-4720-a872-03a1b0b6a8f2 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:52.847782 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1132/2272] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 10 22:52:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:52.855232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04855a4c-90e5-4853-80f3-942119dca77f None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:52.855864 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04855a4c-90e5-4853-80f3-942119dca77f None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:52.856222 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04855a4c-90e5-4853-80f3-942119dca77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:52.880532 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-04855a4c-90e5-4853-80f3-942119dca77f None None] MFA Rules not processed for user `b22838a8ad3949e59cdbaaa1801cd07f`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:52.882035 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-04855a4c-90e5-4853-80f3-942119dca77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:52.894880 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1141/2273] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 10 22:52:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:52:52.907466 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b7691-a3de-4720-a872-03a1b0b6a8f2 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:52.909477 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-ae4b7691-a3de-4720-a872-03a1b0b6a8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:52.978949 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b7691-a3de-4720-a872-03a1b0b6a8f2 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:52.984402 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24654e9-0093-486c-bac0-b9d57e0187bf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc7524edf5d5422f91d92fd91be5ee1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:52.985152 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24654e9-0093-486c-bac0-b9d57e0187bf service glance] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:52.985481 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24654e9-0093-486c-bac0-b9d57e0187bf service glance] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:52.985761 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24654e9-0093-486c-bac0-b9d57e0187bf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:53.012693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24654e9-0093-486c-bac0-b9d57e0187bf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:53.012693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24654e9-0093-486c-bac0-b9d57e0187bf service glance] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:53.013035 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1133/2274] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:52:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:52:53.083413 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46746f08-d408-49f8-b2bb-afd06823f6cb None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:53.083607 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46746f08-d408-49f8-b2bb-afd06823f6cb None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:53.083881 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46746f08-d408-49f8-b2bb-afd06823f6cb None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:53.085231 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1142/2275] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 10 22:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:53.089943 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eaf52c-b110-4928-aadd-49699d48638b None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:53.090256 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eaf52c-b110-4928-aadd-49699d48638b None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:53.090535 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eaf52c-b110-4928-aadd-49699d48638b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:53.113000 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-62eaf52c-b110-4928-aadd-49699d48638b None None] MFA Rules not processed for user `7fcb5a674ff94b3d8f79efbbe4959069`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:53.114261 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-62eaf52c-b110-4928-aadd-49699d48638b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:53.120783 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1134/2276] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 10 22:52:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:52:53.134482 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46746f08-d408-49f8-b2bb-afd06823f6cb None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:53.149687 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46746f08-d408-49f8-b2bb-afd06823f6cb None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:53.155146 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be008d7-7d13-4c6d-9e06-652a6f85234e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:53.155917 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be008d7-7d13-4c6d-9e06-652a6f85234e service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:53.156298 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be008d7-7d13-4c6d-9e06-652a6f85234e service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:53.156669 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be008d7-7d13-4c6d-9e06-652a6f85234e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:53.166729 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be008d7-7d13-4c6d-9e06-652a6f85234e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:53.168834 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be008d7-7d13-4c6d-9e06-652a6f85234e service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:53.179871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1143/2277] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:52:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:52:53.264517 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62607f24-3525-45e7-8683-2c696cfba581 None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:53.265008 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62607f24-3525-45e7-8683-2c696cfba581 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:53.265106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62607f24-3525-45e7-8683-2c696cfba581 None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:53.266475 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1135/2278] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 10 22:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:52:53.272189 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ee41f-bdb8-4d00-9f78-a278c4272f7d None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:53.272471 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ee41f-bdb8-4d00-9f78-a278c4272f7d None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:53.272745 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ee41f-bdb8-4d00-9f78-a278c4272f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:53.298185 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-643ee41f-bdb8-4d00-9f78-a278c4272f7d None None] MFA Rules not processed for user `b22838a8ad3949e59cdbaaa1801cd07f`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:52:53.299382 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-643ee41f-bdb8-4d00-9f78-a278c4272f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:53.310025 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1144/2279] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 10 22:52:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:52:53.320996 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62607f24-3525-45e7-8683-2c696cfba581 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:53.332590 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62607f24-3525-45e7-8683-2c696cfba581 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:53.336361 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c2673b-5ec3-4de0-810d-2f638d0aa754 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:53.336952 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2673b-5ec3-4de0-810d-2f638d0aa754 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:53.337228 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2673b-5ec3-4de0-810d-2f638d0aa754 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:53.337527 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2673b-5ec3-4de0-810d-2f638d0aa754 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:53.348624 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c2673b-5ec3-4de0-810d-2f638d0aa754 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:53.350102 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c2673b-5ec3-4de0-810d-2f638d0aa754 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:53.362248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1136/2280] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:52:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:52:53.430797 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ee41f-bdb8-4d00-9f78-a278c4272f7d None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:52:53.441434 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-643ee41f-bdb8-4d00-9f78-a278c4272f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:53.543906 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ee41f-bdb8-4d00-9f78-a278c4272f7d None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:52:53.549326 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0067556e-96e0-4638-8842-e7dc307e5e3c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f46dee0f73e49eab54c11556e3b86bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:52:53.549934 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0067556e-96e0-4638-8842-e7dc307e5e3c service placement] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:52:53.550212 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0067556e-96e0-4638-8842-e7dc307e5e3c service placement] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:52:53.550483 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0067556e-96e0-4638-8842-e7dc307e5e3c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:52:53.552205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-0067556e-96e0-4638-8842-e7dc307e5e3c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:52:53.563858 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0067556e-96e0-4638-8842-e7dc307e5e3c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:52:53.565458 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0067556e-96e0-4638-8842-e7dc307e5e3c service placement] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:52:53.575138 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1145/2281] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:52:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:53:00.974324 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90945981-4244-4efd-9468-04c8c826f4d2 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:00.974766 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90945981-4244-4efd-9468-04c8c826f4d2 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:00.974896 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90945981-4244-4efd-9468-04c8c826f4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:00.993990 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-90945981-4244-4efd-9468-04c8c826f4d2 None None] MFA Rules not processed for user `aff5e804c64345b298c745287a1d0a32`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:00.995196 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-90945981-4244-4efd-9468-04c8c826f4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:01.002974 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1137/2282] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 10 22:53:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:53:01.015676 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0067556e-96e0-4638-8842-e7dc307e5e3c service placement] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:01.026869 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0067556e-96e0-4638-8842-e7dc307e5e3c service placement] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:01.031925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b6d84-defb-4569-accf-282781cdd1d0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e3a76df95a34b9fb9816f22b85d1071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:01.032512 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b6d84-defb-4569-accf-282781cdd1d0 service swift] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:01.032788 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b6d84-defb-4569-accf-282781cdd1d0 service swift] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:01.033052 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b6d84-defb-4569-accf-282781cdd1d0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:01.041261 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b6d84-defb-4569-accf-282781cdd1d0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:01.042885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b6d84-defb-4569-accf-282781cdd1d0 service swift] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:01.051972 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1146/2283] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 10 22:53:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 22:53:09.074622 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90945981-4244-4efd-9468-04c8c826f4d2 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:09.090047 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90945981-4244-4efd-9468-04c8c826f4d2 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:09.100181 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5efbe72-0a93-4ea9-ae06-b836f654ec07 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f46dee0f73e49eab54c11556e3b86bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:09.101240 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efbe72-0a93-4ea9-ae06-b836f654ec07 service placement] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:09.101785 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efbe72-0a93-4ea9-ae06-b836f654ec07 service placement] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:09.102181 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efbe72-0a93-4ea9-ae06-b836f654ec07 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:09.104789 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-b5efbe72-0a93-4ea9-ae06-b836f654ec07 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:09.124936 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5efbe72-0a93-4ea9-ae06-b836f654ec07 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:09.126984 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5efbe72-0a93-4ea9-ae06-b836f654ec07 service placement] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:09.134830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b6d84-defb-4569-accf-282781cdd1d0 service swift] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:09.137007 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-a28b6d84-defb-4569-accf-282781cdd1d0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:09.141099 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1138/2284] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:53:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:53:09.206967 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:09.234473 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b6d84-defb-4569-accf-282781cdd1d0 service swift] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:09.239410 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68652f48-1290-4265-a388-5d440e28289e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:09.240041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68652f48-1290-4265-a388-5d440e28289e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:09.240339 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68652f48-1290-4265-a388-5d440e28289e admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:09.240593 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68652f48-1290-4265-a388-5d440e28289e admin admin] PATH_INFO: `/v3/users/0f3c2f531a5942e6afabb16521523754` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:09.241756 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68652f48-1290-4265-a388-5d440e28289e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f3c2f531a5942e6afabb16521523754)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:09.243691 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68652f48-1290-4265-a388-5d440e28289e admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:09.293247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-68652f48-1290-4265-a388-5d440e28289e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f3c2f531a5942e6afabb16521523754'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:53:09.302154 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-68652f48-1290-4265-a388-5d440e28289e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f3c2f531a5942e6afabb16521523754'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:53:09.311279 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-68652f48-1290-4265-a388-5d440e28289e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f3c2f531a5942e6afabb16521523754'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:53:09.312616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-68652f48-1290-4265-a388-5d440e28289e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f3c2f531a5942e6afabb16521523754'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:53:09.313446 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-68652f48-1290-4265-a388-5d440e28289e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f3c2f531a5942e6afabb16521523754'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:53:09.331866 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1147/2285] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:53:09 2022] DELETE /identity/v3/users/0f3c2f531a5942e6afabb16521523754 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:53:09.795290 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5efbe72-0a93-4ea9-ae06-b836f654ec07 service placement] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:09.807145 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5efbe72-0a93-4ea9-ae06-b836f654ec07 service placement] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:09.873544 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de02c6-a43a-48cd-bb4a-d1d6243baa09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:09.874181 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de02c6-a43a-48cd-bb4a-d1d6243baa09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:09.874490 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de02c6-a43a-48cd-bb4a-d1d6243baa09 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:09.874762 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de02c6-a43a-48cd-bb4a-d1d6243baa09 admin admin] PATH_INFO: `/v3/projects/c38d0effe2b242bbb7ea221ec2da0177` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:09.875880 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93de02c6-a43a-48cd-bb4a-d1d6243baa09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c38d0effe2b242bbb7ea221ec2da0177)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:09.877836 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93de02c6-a43a-48cd-bb4a-d1d6243baa09 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:09.927845 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-93de02c6-a43a-48cd-bb4a-d1d6243baa09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c38d0effe2b242bbb7ea221ec2da0177', 'request_id': 'req-93de02c6-a43a-48cd-bb4a-d1d6243baa09'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:53:09.931045 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:09.937561 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-93de02c6-a43a-48cd-bb4a-d1d6243baa09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c38d0effe2b242bbb7ea221ec2da0177', 'request_id': 'req-93de02c6-a43a-48cd-bb4a-d1d6243baa09'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:53:09.944162 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-93de02c6-a43a-48cd-bb4a-d1d6243baa09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c38d0effe2b242bbb7ea221ec2da0177', 'request_id': 'req-93de02c6-a43a-48cd-bb4a-d1d6243baa09'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:53:09.955929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-93de02c6-a43a-48cd-bb4a-d1d6243baa09 admin admin] The token cache is being invalidate because project c38d0effe2b242bbb7ea221ec2da0177 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:53:09.956161 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-93de02c6-a43a-48cd-bb4a-d1d6243baa09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:53:09.956803 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-93de02c6-a43a-48cd-bb4a-d1d6243baa09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:53:09.958329 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1139/2286] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:53:09 2022] DELETE /identity/v3/projects/c38d0effe2b242bbb7ea221ec2da0177 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:53:09.963587 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21691ce4-2e04-4fbc-b285-6848d43e8b1b None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:09.963877 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21691ce4-2e04-4fbc-b285-6848d43e8b1b None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:09.964154 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21691ce4-2e04-4fbc-b285-6848d43e8b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:09.986264 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-21691ce4-2e04-4fbc-b285-6848d43e8b1b None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:09.987525 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-21691ce4-2e04-4fbc-b285-6848d43e8b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:10.054246 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1148/2287] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:53:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:53:10.063241 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c40f4-4538-4017-9a38-7371cf4ba8b3 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:10.063528 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c40f4-4538-4017-9a38-7371cf4ba8b3 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:10.063809 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c40f4-4538-4017-9a38-7371cf4ba8b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:10.084887 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-2e9c40f4-4538-4017-9a38-7371cf4ba8b3 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:10.086172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-2e9c40f4-4538-4017-9a38-7371cf4ba8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:10.095302 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1140/2288] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:53:10.099567 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21691ce4-2e04-4fbc-b285-6848d43e8b1b None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:10.109712 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21691ce4-2e04-4fbc-b285-6848d43e8b1b None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:10.114414 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca26eb-da99-4379-8eb9-713494fe0041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:10.115038 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca26eb-da99-4379-8eb9-713494fe0041 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:10.115318 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca26eb-da99-4379-8eb9-713494fe0041 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:10.115590 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca26eb-da99-4379-8eb9-713494fe0041 admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:10.116242 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca26eb-da99-4379-8eb9-713494fe0041 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:53:10.116543 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca26eb-da99-4379-8eb9-713494fe0041 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:10.118189 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca26eb-da99-4379-8eb9-713494fe0041 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:10.136996 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c40f4-4538-4017-9a38-7371cf4ba8b3 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:10.141532 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1149/2289] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:53:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:53:10.148571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c40f4-4538-4017-9a38-7371cf4ba8b3 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:10.154697 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c0718-7405-47d3-81a2-c5b81cb51df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:10.155950 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c0718-7405-47d3-81a2-c5b81cb51df0 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:10.155950 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c0718-7405-47d3-81a2-c5b81cb51df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:10.155950 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c0718-7405-47d3-81a2-c5b81cb51df0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:10.156542 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616c0718-7405-47d3-81a2-c5b81cb51df0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:10.159218 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616c0718-7405-47d3-81a2-c5b81cb51df0 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:10.191759 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1141/2290] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:53:10 2022] POST /identity/v3/projects => generated 391 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:53:10.196018 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca26eb-da99-4379-8eb9-713494fe0041 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:10.210628 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca26eb-da99-4379-8eb9-713494fe0041 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:10.255434 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:10.274765 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9358ed-94c1-4482-9ba1-c92ec73c293e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:10.275419 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9358ed-94c1-4482-9ba1-c92ec73c293e admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:10.275728 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9358ed-94c1-4482-9ba1-c92ec73c293e admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:10.276000 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9358ed-94c1-4482-9ba1-c92ec73c293e admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:10.276617 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9358ed-94c1-4482-9ba1-c92ec73c293e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:10.279156 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9358ed-94c1-4482-9ba1-c92ec73c293e admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:10.305147 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1150/2291] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:53:10 2022] POST /identity/v3/users => generated 296 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:53:10.306557 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c0718-7405-47d3-81a2-c5b81cb51df0 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:10.317489 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c0718-7405-47d3-81a2-c5b81cb51df0 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:10.322632 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e2e1b-37e9-4e17-884c-68d611802f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:10.323240 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e2e1b-37e9-4e17-884c-68d611802f91 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:10.323572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e2e1b-37e9-4e17-884c-68d611802f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:10.323994 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e2e1b-37e9-4e17-884c-68d611802f91 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:10.324684 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9e2e1b-37e9-4e17-884c-68d611802f91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:10.326953 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9e2e1b-37e9-4e17-884c-68d611802f91 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:10.338194 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1142/2292] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 22:53:10.343151 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9358ed-94c1-4482-9ba1-c92ec73c293e admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:10.353094 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9358ed-94c1-4482-9ba1-c92ec73c293e admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:10.357986 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb7fe2-19fc-46d2-ae7e-80f38dde51a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:10.358585 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb7fe2-19fc-46d2-ae7e-80f38dde51a8 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:10.358874 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb7fe2-19fc-46d2-ae7e-80f38dde51a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:10.359145 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb7fe2-19fc-46d2-ae7e-80f38dde51a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:10.359788 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cb7fe2-19fc-46d2-ae7e-80f38dde51a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:10.361433 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cb7fe2-19fc-46d2-ae7e-80f38dde51a8 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:10.377918 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1151/2293] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:53:10.379570 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e2e1b-37e9-4e17-884c-68d611802f91 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:10.397170 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e2e1b-37e9-4e17-884c-68d611802f91 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:10.404269 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd56a5-5bc8-4fb6-af22-9097abb98dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:10.405670 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd56a5-5bc8-4fb6-af22-9097abb98dc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:10.406207 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd56a5-5bc8-4fb6-af22-9097abb98dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:10.406513 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd56a5-5bc8-4fb6-af22-9097abb98dc1 admin admin] PATH_INFO: `/v3/projects/9e7a81d8aee444de96cca5c330f70557/users/0a3e9299279c44618906252fde5a3925/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:10.426171 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dd56a5-5bc8-4fb6-af22-9097abb98dc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e7a81d8aee444de96cca5c330f70557, user_id=0a3e9299279c44618906252fde5a3925, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:10.429178 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dd56a5-5bc8-4fb6-af22-9097abb98dc1 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:10.444932 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1143/2294] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:53:10 2022] PUT /identity/v3/projects/9e7a81d8aee444de96cca5c330f70557/users/0a3e9299279c44618906252fde5a3925/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:53:10.456736 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a340e-c1a7-471e-8e2a-f7a34a8f3b4e None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:10.457016 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a340e-c1a7-471e-8e2a-f7a34a8f3b4e None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:10.457304 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a340e-c1a7-471e-8e2a-f7a34a8f3b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:10.492108 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-6d9a340e-c1a7-471e-8e2a-f7a34a8f3b4e None None] MFA Rules not processed for user `0a3e9299279c44618906252fde5a3925`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:10.493261 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-6d9a340e-c1a7-471e-8e2a-f7a34a8f3b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:10.561993 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1152/2295] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:53:10 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:53:10.573674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd56a5-5bc8-4fb6-af22-9097abb98dc1 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:10.575739 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-32dd56a5-5bc8-4fb6-af22-9097abb98dc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:10.633641 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd56a5-5bc8-4fb6-af22-9097abb98dc1 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:10.638925 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7393d94-3a25-4fae-b2d6-0670a3f09c50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:10.639963 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7393d94-3a25-4fae-b2d6-0670a3f09c50 service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:10.640452 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7393d94-3a25-4fae-b2d6-0670a3f09c50 service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:10.640936 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7393d94-3a25-4fae-b2d6-0670a3f09c50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:10.652313 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7393d94-3a25-4fae-b2d6-0670a3f09c50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:10.654296 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7393d94-3a25-4fae-b2d6-0670a3f09c50 service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:10.664908 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1144/2296] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:53:10 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:53:20.558600 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=47,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:20.659841 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:DELETE=3,keystone:SELECT=59,keystone:INSERT=2 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:36.570136 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a340e-c1a7-471e-8e2a-f7a34a8f3b4e None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:36.571695 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-6d9a340e-c1a7-471e-8e2a-f7a34a8f3b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:36.648862 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a340e-c1a7-471e-8e2a-f7a34a8f3b4e None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:36.654955 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7fcfe0-2fcb-4e91-8fbc-9c5b8dacac50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:36.655841 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fcfe0-2fcb-4e91-8fbc-9c5b8dacac50 service nova] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:36.656114 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fcfe0-2fcb-4e91-8fbc-9c5b8dacac50 service nova] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:36.656379 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fcfe0-2fcb-4e91-8fbc-9c5b8dacac50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:36.658214 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-fa7fcfe0-2fcb-4e91-8fbc-9c5b8dacac50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:36.671940 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7fcfe0-2fcb-4e91-8fbc-9c5b8dacac50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:36.673501 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7fcfe0-2fcb-4e91-8fbc-9c5b8dacac50 service nova] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:36.684053 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1153/2297] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:53:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:53:38.325789 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7393d94-3a25-4fae-b2d6-0670a3f09c50 service cinder] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:38.390790 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7393d94-3a25-4fae-b2d6-0670a3f09c50 service cinder] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:38.395219 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e0a166-12b2-4e95-b03b-b50c2c59b832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:38.395840 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0a166-12b2-4e95-b03b-b50c2c59b832 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:38.396119 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0a166-12b2-4e95-b03b-b50c2c59b832 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:38.396390 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0a166-12b2-4e95-b03b-b50c2c59b832 admin admin] PATH_INFO: `/v3/users/0a3e9299279c44618906252fde5a3925` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:38.397521 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e0a166-12b2-4e95-b03b-b50c2c59b832 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a3e9299279c44618906252fde5a3925)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:38.399452 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e0a166-12b2-4e95-b03b-b50c2c59b832 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:38.451450 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-98e0a166-12b2-4e95-b03b-b50c2c59b832 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a3e9299279c44618906252fde5a3925'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:53:38.458450 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-98e0a166-12b2-4e95-b03b-b50c2c59b832 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a3e9299279c44618906252fde5a3925'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:53:38.470069 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-98e0a166-12b2-4e95-b03b-b50c2c59b832 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a3e9299279c44618906252fde5a3925'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:53:38.470755 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-98e0a166-12b2-4e95-b03b-b50c2c59b832 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a3e9299279c44618906252fde5a3925'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:53:38.471361 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-98e0a166-12b2-4e95-b03b-b50c2c59b832 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a3e9299279c44618906252fde5a3925'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:53:38.484505 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1145/2298] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:53:38 2022] DELETE /identity/v3/users/0a3e9299279c44618906252fde5a3925 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:53:38.489037 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee3514-5a39-4d99-abcb-80eed5e086e4 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:38.489594 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee3514-5a39-4d99-abcb-80eed5e086e4 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:38.489744 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee3514-5a39-4d99-abcb-80eed5e086e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:38.509065 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-bbee3514-5a39-4d99-abcb-80eed5e086e4 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:38.510303 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-bbee3514-5a39-4d99-abcb-80eed5e086e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:38.570565 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1154/2299] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:53:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:53:38.580660 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e0a166-12b2-4e95-b03b-b50c2c59b832 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:38.582124 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-98e0a166-12b2-4e95-b03b-b50c2c59b832 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:38.638628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e0a166-12b2-4e95-b03b-b50c2c59b832 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:38.642315 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e0f75-cf56-41b7-88af-5ff49db803e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:38.642846 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e0f75-cf56-41b7-88af-5ff49db803e7 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:38.643121 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e0f75-cf56-41b7-88af-5ff49db803e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:38.643390 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e0f75-cf56-41b7-88af-5ff49db803e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:38.652693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360e0f75-cf56-41b7-88af-5ff49db803e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:38.654261 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360e0f75-cf56-41b7-88af-5ff49db803e7 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:38.663919 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1146/2300] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:53:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:53:38.933982 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee3514-5a39-4d99-abcb-80eed5e086e4 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:38.936115 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-bbee3514-5a39-4d99-abcb-80eed5e086e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:38.949117 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee3514-5a39-4d99-abcb-80eed5e086e4 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:38.955216 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf8aa61-6495-4b04-a4d2-5b2101ce6c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:38.955863 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8aa61-6495-4b04-a4d2-5b2101ce6c91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:38.956145 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8aa61-6495-4b04-a4d2-5b2101ce6c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:38.956423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8aa61-6495-4b04-a4d2-5b2101ce6c91 admin admin] PATH_INFO: `/v3/projects/9e7a81d8aee444de96cca5c330f70557` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:38.957589 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf8aa61-6495-4b04-a4d2-5b2101ce6c91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e7a81d8aee444de96cca5c330f70557)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:38.960408 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf8aa61-6495-4b04-a4d2-5b2101ce6c91 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:39.008995 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-bdf8aa61-6495-4b04-a4d2-5b2101ce6c91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e7a81d8aee444de96cca5c330f70557', 'request_id': 'req-bdf8aa61-6495-4b04-a4d2-5b2101ce6c91'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:53:39.015230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-bdf8aa61-6495-4b04-a4d2-5b2101ce6c91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e7a81d8aee444de96cca5c330f70557', 'request_id': 'req-bdf8aa61-6495-4b04-a4d2-5b2101ce6c91'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:53:39.024264 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-bdf8aa61-6495-4b04-a4d2-5b2101ce6c91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e7a81d8aee444de96cca5c330f70557', 'request_id': 'req-bdf8aa61-6495-4b04-a4d2-5b2101ce6c91'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:53:39.037656 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-bdf8aa61-6495-4b04-a4d2-5b2101ce6c91 admin admin] The token cache is being invalidate because project 9e7a81d8aee444de96cca5c330f70557 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:53:39.037949 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-bdf8aa61-6495-4b04-a4d2-5b2101ce6c91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:53:39.038629 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-bdf8aa61-6495-4b04-a4d2-5b2101ce6c91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:53:39.040100 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1155/2301] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:53:38 2022] DELETE /identity/v3/projects/9e7a81d8aee444de96cca5c330f70557 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:53:40.267495 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ff490-4491-4de8-a63a-92b6c64ab6bf None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:40.267971 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ff490-4491-4de8-a63a-92b6c64ab6bf None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:40.268091 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ff490-4491-4de8-a63a-92b6c64ab6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:40.288107 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-597ff490-4491-4de8-a63a-92b6c64ab6bf None None] MFA Rules not processed for user `aff5e804c64345b298c745287a1d0a32`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:53:40.289328 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-597ff490-4491-4de8-a63a-92b6c64ab6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:40.307030 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:40.347961 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1147/2302] 10.222.0.76 () {58 vars in 967 bytes} [Wed Aug 10 22:53:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:53:40.371743 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf8aa61-6495-4b04-a4d2-5b2101ce6c91 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:53:40.373625 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-bdf8aa61-6495-4b04-a4d2-5b2101ce6c91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:53:40.385041 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:40.492375 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf8aa61-6495-4b04-a4d2-5b2101ce6c91 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:53:40.497400 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3657189-9d02-441e-b052-3409fbbfedb8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e3a76df95a34b9fb9816f22b85d1071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:53:40.498550 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3657189-9d02-441e-b052-3409fbbfedb8 service swift] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:53:40.498828 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3657189-9d02-441e-b052-3409fbbfedb8 service swift] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:53:40.499097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3657189-9d02-441e-b052-3409fbbfedb8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:53:40.508460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3657189-9d02-441e-b052-3409fbbfedb8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:53:40.510108 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3657189-9d02-441e-b052-3409fbbfedb8 service swift] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:53:40.519937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1156/2303] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 10 22:53:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 22:53:50.340272 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=15 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:53:50.515986 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=28 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:54:42.201272 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ff490-4491-4de8-a63a-92b6c64ab6bf None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:54:42.203035 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-597ff490-4491-4de8-a63a-92b6c64ab6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:54:42.276418 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ff490-4491-4de8-a63a-92b6c64ab6bf None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:54:42.282313 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af1571-a71c-4ff4-ad02-de9cb9e50185 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:54:42.282917 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af1571-a71c-4ff4-ad02-de9cb9e50185 service nova] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:54:42.283192 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af1571-a71c-4ff4-ad02-de9cb9e50185 service nova] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:54:42.283460 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af1571-a71c-4ff4-ad02-de9cb9e50185 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:54:42.285154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-20af1571-a71c-4ff4-ad02-de9cb9e50185 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:54:42.344413 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20af1571-a71c-4ff4-ad02-de9cb9e50185 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:54:42.346038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20af1571-a71c-4ff4-ad02-de9cb9e50185 service nova] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:54:42.355506 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1148/2304] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:54:42 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:54:52.351100 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=56 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:55:04.676324 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3657189-9d02-441e-b052-3409fbbfedb8 service swift] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:55:04.692481 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3657189-9d02-441e-b052-3409fbbfedb8 service swift] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:55:04.698906 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74800eb0-7b32-49b8-a94f-452e5484e2ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:55:04.700471 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74800eb0-7b32-49b8-a94f-452e5484e2ba service nova] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:55:04.700896 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74800eb0-7b32-49b8-a94f-452e5484e2ba service nova] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:55:04.701240 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74800eb0-7b32-49b8-a94f-452e5484e2ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:55:04.704466 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-74800eb0-7b32-49b8-a94f-452e5484e2ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:55:04.721320 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74800eb0-7b32-49b8-a94f-452e5484e2ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:55:04.723236 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74800eb0-7b32-49b8-a94f-452e5484e2ba service nova] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:55:04.735799 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1157/2305] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:55:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:55:11.897736 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e0224-c387-4287-8296-8b6f687c3d43 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:55:11.898466 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e0224-c387-4287-8296-8b6f687c3d43 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:55:11.898466 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e0224-c387-4287-8296-8b6f687c3d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:55:11.921508 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-2e4e0224-c387-4287-8296-8b6f687c3d43 None None] MFA Rules not processed for user `6367371496f04cc2a7c105fbbed68dca`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:55:11.922717 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-2e4e0224-c387-4287-8296-8b6f687c3d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:55:11.929990 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1149/2306] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:55:11 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:55:14.731055 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=6 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:55:15.097243 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae0b10-7129-4911-85f5-7484781a555a None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:55:15.097631 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae0b10-7129-4911-85f5-7484781a555a None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:55:15.097989 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae0b10-7129-4911-85f5-7484781a555a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:55:15.117306 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-0eae0b10-7129-4911-85f5-7484781a555a None None] MFA Rules not processed for user `12e6b0b71ecb4825aeab6735eb44e63e`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:55:15.118526 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-0eae0b10-7129-4911-85f5-7484781a555a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:55:15.182494 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1158/2307] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:55:15 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:55:19.815266 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42297eef-fa03-4ff3-b270-24d175d4640b None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:55:19.815802 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42297eef-fa03-4ff3-b270-24d175d4640b None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:55:19.815802 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42297eef-fa03-4ff3-b270-24d175d4640b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:55:19.835619 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-42297eef-fa03-4ff3-b270-24d175d4640b None None] MFA Rules not processed for user `da22fc66a1d14fa983463331de110258`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:55:19.836752 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-42297eef-fa03-4ff3-b270-24d175d4640b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:55:19.896893 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1150/2308] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:55:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:55:20.202741 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae0b10-7129-4911-85f5-7484781a555a None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:55:20.204203 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-0eae0b10-7129-4911-85f5-7484781a555a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:55:20.272081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae0b10-7129-4911-85f5-7484781a555a None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:55:20.276132 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931910bb-5fce-44ff-ba39-7397b39a92fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:55:20.277054 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931910bb-5fce-44ff-ba39-7397b39a92fd service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:55:20.277363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931910bb-5fce-44ff-ba39-7397b39a92fd service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:55:20.277644 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931910bb-5fce-44ff-ba39-7397b39a92fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:55:20.279612 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-931910bb-5fce-44ff-ba39-7397b39a92fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:55:20.289649 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931910bb-5fce-44ff-ba39-7397b39a92fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:55:20.291084 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931910bb-5fce-44ff-ba39-7397b39a92fd service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:55:20.300416 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1159/2309] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:55:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:55:24.799040 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42297eef-fa03-4ff3-b270-24d175d4640b None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:55:24.800457 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-42297eef-fa03-4ff3-b270-24d175d4640b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:55:24.818422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42297eef-fa03-4ff3-b270-24d175d4640b None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:55:24.827445 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30180e6-f680-4fd7-9058-a1641809f15b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:55:24.828196 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30180e6-f680-4fd7-9058-a1641809f15b service nova] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:55:24.830302 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30180e6-f680-4fd7-9058-a1641809f15b service nova] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:55:24.830575 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30180e6-f680-4fd7-9058-a1641809f15b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:55:24.832361 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-c30180e6-f680-4fd7-9058-a1641809f15b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:55:24.845726 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30180e6-f680-4fd7-9058-a1641809f15b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:55:24.847331 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30180e6-f680-4fd7-9058-a1641809f15b service nova] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:55:24.869527 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1151/2310] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:55:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 22:55:30.296665 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=51 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:55:34.854917 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=34 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:55:40.318055 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931910bb-5fce-44ff-ba39-7397b39a92fd service neutron] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:55:40.319455 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-931910bb-5fce-44ff-ba39-7397b39a92fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:55:40.460050 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931910bb-5fce-44ff-ba39-7397b39a92fd service neutron] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:55:40.466015 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc8e04-fa18-4068-b526-70af7ba94e4c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f46dee0f73e49eab54c11556e3b86bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:55:40.466618 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc8e04-fa18-4068-b526-70af7ba94e4c service placement] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:55:40.466893 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc8e04-fa18-4068-b526-70af7ba94e4c service placement] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:55:40.467162 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc8e04-fa18-4068-b526-70af7ba94e4c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:55:40.468914 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-49fc8e04-fa18-4068-b526-70af7ba94e4c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:55:40.480149 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fc8e04-fa18-4068-b526-70af7ba94e4c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:55:40.482427 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fc8e04-fa18-4068-b526-70af7ba94e4c service placement] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:55:40.492038 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1160/2311] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:55:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:55:50.487985 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=63 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:55:55.466517 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30180e6-f680-4fd7-9058-a1641809f15b service nova] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:55:55.536344 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30180e6-f680-4fd7-9058-a1641809f15b service nova] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:55:55.542100 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201bca5-796a-4fb2-afc3-beaa12ee086e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:55:55.542738 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201bca5-796a-4fb2-afc3-beaa12ee086e service nova] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:55:55.543020 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201bca5-796a-4fb2-afc3-beaa12ee086e service nova] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:55:55.543291 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201bca5-796a-4fb2-afc3-beaa12ee086e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:55:55.545117 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-9201bca5-796a-4fb2-afc3-beaa12ee086e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:55:55.606421 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9201bca5-796a-4fb2-afc3-beaa12ee086e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:55:55.608042 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9201bca5-796a-4fb2-afc3-beaa12ee086e service nova] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:55:55.619168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1152/2312] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:55:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:55:59.825603 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e67cd-6a81-4d68-b4db-235deb7ffcca None None] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:55:59.826182 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e67cd-6a81-4d68-b4db-235deb7ffcca None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:55:59.826995 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e67cd-6a81-4d68-b4db-235deb7ffcca None None] PATH_INFO: `/` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:55:59.827936 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1161/2313] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 10 22:55:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:55:59.835176 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853d6e6-3fc1-43c6-a42e-5fb4565ced00 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:55:59.835743 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853d6e6-3fc1-43c6-a42e-5fb4565ced00 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:55:59.835743 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853d6e6-3fc1-43c6-a42e-5fb4565ced00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:55:59.898952 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-0853d6e6-3fc1-43c6-a42e-5fb4565ced00 None None] MFA Rules not processed for user `7fcb5a674ff94b3d8f79efbbe4959069`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:55:59.900179 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-0853d6e6-3fc1-43c6-a42e-5fb4565ced00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:55:59.920258 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1153/2314] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 10 22:55:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:55:59.936851 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2e67cd-6a81-4d68-b4db-235deb7ffcca None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:55:59.939488 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-1f2e67cd-6a81-4d68-b4db-235deb7ffcca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:55:59.954028 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2e67cd-6a81-4d68-b4db-235deb7ffcca None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:55:59.962146 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642ed994-4004-4be0-b3f3-8b3d4ce6043f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:55:59.965518 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ed994-4004-4be0-b3f3-8b3d4ce6043f service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:55:59.966385 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ed994-4004-4be0-b3f3-8b3d4ce6043f service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:55:59.966385 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ed994-4004-4be0-b3f3-8b3d4ce6043f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:55:59.975092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642ed994-4004-4be0-b3f3-8b3d4ce6043f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:55:59.976634 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642ed994-4004-4be0-b3f3-8b3d4ce6043f service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:55:59.984509 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1162/2315] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:55:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:56:00.076253 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d793eb9-dd1b-44db-8ca6-cbec3d931e4c None None] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:00.076730 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d793eb9-dd1b-44db-8ca6-cbec3d931e4c None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:00.077005 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d793eb9-dd1b-44db-8ca6-cbec3d931e4c None None] PATH_INFO: `/` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:00.078551 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1154/2316] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 10 22:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 22:56:00.084748 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd85bf-f143-47d0-aac3-05ad9935c21b None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:00.085033 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd85bf-f143-47d0-aac3-05ad9935c21b None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:00.085330 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd85bf-f143-47d0-aac3-05ad9935c21b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:00.118145 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-afcd85bf-f143-47d0-aac3-05ad9935c21b None None] MFA Rules not processed for user `b22838a8ad3949e59cdbaaa1801cd07f`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:56:00.119359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-afcd85bf-f143-47d0-aac3-05ad9935c21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:56:00.129596 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1163/2317] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 10 22:56:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:56:00.140121 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d793eb9-dd1b-44db-8ca6-cbec3d931e4c None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:00.151916 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d793eb9-dd1b-44db-8ca6-cbec3d931e4c None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:00.155328 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b0ecd-4614-400d-8580-48a63670970e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:00.155948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b0ecd-4614-400d-8580-48a63670970e service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:00.156218 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b0ecd-4614-400d-8580-48a63670970e service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:00.156500 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b0ecd-4614-400d-8580-48a63670970e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:00.167009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040b0ecd-4614-400d-8580-48a63670970e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:00.168424 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040b0ecd-4614-400d-8580-48a63670970e service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:00.179698 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1155/2318] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:56:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:56:06.347786 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114cfa5-a39e-42e0-b3f4-757d7ccc6612 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:06.348283 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114cfa5-a39e-42e0-b3f4-757d7ccc6612 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:06.348379 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114cfa5-a39e-42e0-b3f4-757d7ccc6612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:06.368721 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-0114cfa5-a39e-42e0-b3f4-757d7ccc6612 None None] MFA Rules not processed for user `aff5e804c64345b298c745287a1d0a32`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:56:06.369962 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-0114cfa5-a39e-42e0-b3f4-757d7ccc6612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:56:06.377844 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1164/2319] 10.222.0.76 () {58 vars in 967 bytes} [Wed Aug 10 22:56:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:56:06.427020 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b0ecd-4614-400d-8580-48a63670970e service neutron] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:06.438369 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b0ecd-4614-400d-8580-48a63670970e service neutron] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:06.443327 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d3162-a6c4-43da-9185-0d004fc825f4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e3a76df95a34b9fb9816f22b85d1071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:06.443930 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d3162-a6c4-43da-9185-0d004fc825f4 service swift] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:06.444206 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d3162-a6c4-43da-9185-0d004fc825f4 service swift] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:06.444474 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d3162-a6c4-43da-9185-0d004fc825f4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:06.452470 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73d3162-a6c4-43da-9185-0d004fc825f4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:06.454282 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73d3162-a6c4-43da-9185-0d004fc825f4 service swift] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:06.462465 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1156/2320] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 10 22:56:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 22:56:06.807295 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0114cfa5-a39e-42e0-b3f4-757d7ccc6612 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:06.816738 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0114cfa5-a39e-42e0-b3f4-757d7ccc6612 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:06.820316 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b47247-54b1-4e20-80a7-2e6d1c4545f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:06.820917 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b47247-54b1-4e20-80a7-2e6d1c4545f9 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:06.821203 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b47247-54b1-4e20-80a7-2e6d1c4545f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:06.821516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b47247-54b1-4e20-80a7-2e6d1c4545f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:06.823248 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-32b47247-54b1-4e20-80a7-2e6d1c4545f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:56:06.908678 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b47247-54b1-4e20-80a7-2e6d1c4545f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:06.910129 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b47247-54b1-4e20-80a7-2e6d1c4545f9 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:06.928532 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1165/2321] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:56:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:56:06.936372 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d3162-a6c4-43da-9185-0d004fc825f4 service swift] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:06.949626 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d3162-a6c4-43da-9185-0d004fc825f4 service swift] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:06.956329 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfad367-2945-4200-8d8b-9cc54507fc07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:06.956924 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfad367-2945-4200-8d8b-9cc54507fc07 service nova] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:06.957208 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfad367-2945-4200-8d8b-9cc54507fc07 service nova] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:06.957510 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfad367-2945-4200-8d8b-9cc54507fc07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:06.959219 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-7bfad367-2945-4200-8d8b-9cc54507fc07 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:56:06.993140 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:07.021511 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfad367-2945-4200-8d8b-9cc54507fc07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:07.023232 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfad367-2945-4200-8d8b-9cc54507fc07 service nova] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:07.033678 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1157/2322] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:56:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:56:13.861991 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acc857-5c17-4d38-a5c9-584970628727 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:13.862769 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acc857-5c17-4d38-a5c9-584970628727 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:13.862933 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acc857-5c17-4d38-a5c9-584970628727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:13.895447 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-43acc857-5c17-4d38-a5c9-584970628727 None None] MFA Rules not processed for user `405738ed219b4fdc8e3c07706a13e5a2`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:56:13.896776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-43acc857-5c17-4d38-a5c9-584970628727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:56:13.976204 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1166/2323] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:56:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:56:15.652782 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfad367-2945-4200-8d8b-9cc54507fc07 service nova] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:15.654647 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-7bfad367-2945-4200-8d8b-9cc54507fc07 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:56:15.675046 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfad367-2945-4200-8d8b-9cc54507fc07 service nova] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:15.688321 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c9ea9-b88a-4f5a-8dc6-fecd03b69839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:15.688991 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c9ea9-b88a-4f5a-8dc6-fecd03b69839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:15.689272 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c9ea9-b88a-4f5a-8dc6-fecd03b69839 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:15.689776 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c9ea9-b88a-4f5a-8dc6-fecd03b69839 admin admin] PATH_INFO: `/v3/users/b9f5b7c7c05c42eda7926cd03a0fd9bc` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:15.691136 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400c9ea9-b88a-4f5a-8dc6-fecd03b69839 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9f5b7c7c05c42eda7926cd03a0fd9bc)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:15.693317 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400c9ea9-b88a-4f5a-8dc6-fecd03b69839 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:15.774568 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-400c9ea9-b88a-4f5a-8dc6-fecd03b69839 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9f5b7c7c05c42eda7926cd03a0fd9bc'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:56:15.784640 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-400c9ea9-b88a-4f5a-8dc6-fecd03b69839 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9f5b7c7c05c42eda7926cd03a0fd9bc'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:56:15.807577 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-400c9ea9-b88a-4f5a-8dc6-fecd03b69839 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9f5b7c7c05c42eda7926cd03a0fd9bc'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:56:15.808970 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-400c9ea9-b88a-4f5a-8dc6-fecd03b69839 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9f5b7c7c05c42eda7926cd03a0fd9bc'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:56:15.809797 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-400c9ea9-b88a-4f5a-8dc6-fecd03b69839 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9f5b7c7c05c42eda7926cd03a0fd9bc'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:56:15.838086 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1158/2324] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:56:15 2022] DELETE /identity/v3/users/b9f5b7c7c05c42eda7926cd03a0fd9bc => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:56:16.174106 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43acc857-5c17-4d38-a5c9-584970628727 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:16.199063 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43acc857-5c17-4d38-a5c9-584970628727 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:16.252178 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:16.286889 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241bf50-7793-43cf-b2d8-bb4a84341a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:16.287622 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241bf50-7793-43cf-b2d8-bb4a84341a12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:16.287972 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241bf50-7793-43cf-b2d8-bb4a84341a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:16.288311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241bf50-7793-43cf-b2d8-bb4a84341a12 admin admin] PATH_INFO: `/v3/projects/0f4c72696c814e6e816fd9240569e0db` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:16.289539 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2241bf50-7793-43cf-b2d8-bb4a84341a12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f4c72696c814e6e816fd9240569e0db)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:16.291506 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2241bf50-7793-43cf-b2d8-bb4a84341a12 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:16.337387 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-2241bf50-7793-43cf-b2d8-bb4a84341a12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f4c72696c814e6e816fd9240569e0db', 'request_id': 'req-2241bf50-7793-43cf-b2d8-bb4a84341a12'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:56:16.343175 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-2241bf50-7793-43cf-b2d8-bb4a84341a12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f4c72696c814e6e816fd9240569e0db', 'request_id': 'req-2241bf50-7793-43cf-b2d8-bb4a84341a12'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:56:16.350997 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-2241bf50-7793-43cf-b2d8-bb4a84341a12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f4c72696c814e6e816fd9240569e0db', 'request_id': 'req-2241bf50-7793-43cf-b2d8-bb4a84341a12'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:56:16.364887 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-2241bf50-7793-43cf-b2d8-bb4a84341a12 admin admin] The token cache is being invalidate because project 0f4c72696c814e6e816fd9240569e0db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:56:16.365090 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-2241bf50-7793-43cf-b2d8-bb4a84341a12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:56:16.365786 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-2241bf50-7793-43cf-b2d8-bb4a84341a12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:56:16.367945 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1167/2325] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:56:16 2022] DELETE /identity/v3/projects/0f4c72696c814e6e816fd9240569e0db => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:56:16.371461 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c9ea9-b88a-4f5a-8dc6-fecd03b69839 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:16.372925 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-400c9ea9-b88a-4f5a-8dc6-fecd03b69839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:56:16.387214 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c9ea9-b88a-4f5a-8dc6-fecd03b69839 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:16.462712 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cbcf3b-34b6-42b4-a791-9460281b208e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:16.463175 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbcf3b-34b6-42b4-a791-9460281b208e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:16.463564 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbcf3b-34b6-42b4-a791-9460281b208e admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:16.463944 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbcf3b-34b6-42b4-a791-9460281b208e admin admin] PATH_INFO: `/v3/users/988cb2ebefcd4620b8bbdc6d0647dc7f` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:16.465196 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cbcf3b-34b6-42b4-a791-9460281b208e admin admin] RBAC: Authorizing `identity:delete_user(user_id=988cb2ebefcd4620b8bbdc6d0647dc7f)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:16.467351 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cbcf3b-34b6-42b4-a791-9460281b208e admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:16.522239 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-16cbcf3b-34b6-42b4-a791-9460281b208e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '988cb2ebefcd4620b8bbdc6d0647dc7f'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:56:16.523547 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:16.531763 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-16cbcf3b-34b6-42b4-a791-9460281b208e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '988cb2ebefcd4620b8bbdc6d0647dc7f'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:56:16.544006 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-16cbcf3b-34b6-42b4-a791-9460281b208e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '988cb2ebefcd4620b8bbdc6d0647dc7f'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:56:16.544671 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-16cbcf3b-34b6-42b4-a791-9460281b208e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '988cb2ebefcd4620b8bbdc6d0647dc7f'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:56:16.545380 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-16cbcf3b-34b6-42b4-a791-9460281b208e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '988cb2ebefcd4620b8bbdc6d0647dc7f'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:56:16.561182 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1159/2326] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:56:16 2022] DELETE /identity/v3/users/988cb2ebefcd4620b8bbdc6d0647dc7f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:56:16.742452 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241bf50-7793-43cf-b2d8-bb4a84341a12 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:16.744313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-2241bf50-7793-43cf-b2d8-bb4a84341a12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:56:16.769781 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241bf50-7793-43cf-b2d8-bb4a84341a12 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:16.836042 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef7c15a-218d-4c19-bef0-34973eabac6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:16.836683 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7c15a-218d-4c19-bef0-34973eabac6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:16.836969 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7c15a-218d-4c19-bef0-34973eabac6d admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:16.837241 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7c15a-218d-4c19-bef0-34973eabac6d admin admin] PATH_INFO: `/v3/projects/bea0c702d60148f5b6f225f16c760e71` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:16.838395 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef7c15a-218d-4c19-bef0-34973eabac6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bea0c702d60148f5b6f225f16c760e71)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:16.840318 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef7c15a-218d-4c19-bef0-34973eabac6d admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:16.900216 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-9ef7c15a-218d-4c19-bef0-34973eabac6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bea0c702d60148f5b6f225f16c760e71', 'request_id': 'req-9ef7c15a-218d-4c19-bef0-34973eabac6d'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:56:16.909604 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-9ef7c15a-218d-4c19-bef0-34973eabac6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bea0c702d60148f5b6f225f16c760e71', 'request_id': 'req-9ef7c15a-218d-4c19-bef0-34973eabac6d'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:56:16.929566 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-9ef7c15a-218d-4c19-bef0-34973eabac6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bea0c702d60148f5b6f225f16c760e71', 'request_id': 'req-9ef7c15a-218d-4c19-bef0-34973eabac6d'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:56:16.931004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:16.946378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-9ef7c15a-218d-4c19-bef0-34973eabac6d admin admin] The token cache is being invalidate because project bea0c702d60148f5b6f225f16c760e71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:56:16.946709 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-9ef7c15a-218d-4c19-bef0-34973eabac6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:56:16.947396 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-9ef7c15a-218d-4c19-bef0-34973eabac6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:56:16.950121 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1168/2327] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:56:16 2022] DELETE /identity/v3/projects/bea0c702d60148f5b6f225f16c760e71 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:56:26.557591 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=12,keystone:DELETE=3,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:26.944238 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:50.489479 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cbcf3b-34b6-42b4-a791-9460281b208e admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:50.490957 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-16cbcf3b-34b6-42b4-a791-9460281b208e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:56:50.548653 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cbcf3b-34b6-42b4-a791-9460281b208e admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:50.596165 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791fd535-18f3-4b61-be19-15eecffeee48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:50.596769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fd535-18f3-4b61-be19-15eecffeee48 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:50.597051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fd535-18f3-4b61-be19-15eecffeee48 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:50.597360 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fd535-18f3-4b61-be19-15eecffeee48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:50.599307 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-791fd535-18f3-4b61-be19-15eecffeee48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:56:50.664263 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791fd535-18f3-4b61-be19-15eecffeee48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:50.665731 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791fd535-18f3-4b61-be19-15eecffeee48 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:50.675719 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1160/2328] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:56:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:56:53.915353 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef7c15a-218d-4c19-bef0-34973eabac6d admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:53.916945 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-9ef7c15a-218d-4c19-bef0-34973eabac6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:56:53.983822 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791fd535-18f3-4b61-be19-15eecffeee48 service neutron] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:53.986021 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-791fd535-18f3-4b61-be19-15eecffeee48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:56:54.002959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791fd535-18f3-4b61-be19-15eecffeee48 service neutron] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:54.005263 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef7c15a-218d-4c19-bef0-34973eabac6d admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:54.008400 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d897ac-abc9-4f08-ae52-f90773724243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:54.009065 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d897ac-abc9-4f08-ae52-f90773724243 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:54.009376 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d897ac-abc9-4f08-ae52-f90773724243 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:54.009655 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d897ac-abc9-4f08-ae52-f90773724243 admin admin] PATH_INFO: `/v3/users/6367371496f04cc2a7c105fbbed68dca` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:54.010743 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca6a37-5734-4798-9c68-1da4bf0a4572 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f46dee0f73e49eab54c11556e3b86bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:54.010878 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d897ac-abc9-4f08-ae52-f90773724243 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6367371496f04cc2a7c105fbbed68dca)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:54.011355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca6a37-5734-4798-9c68-1da4bf0a4572 service placement] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:54.011640 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca6a37-5734-4798-9c68-1da4bf0a4572 service placement] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:54.011917 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca6a37-5734-4798-9c68-1da4bf0a4572 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:54.012972 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d897ac-abc9-4f08-ae52-f90773724243 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:54.014016 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-37ca6a37-5734-4798-9c68-1da4bf0a4572 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:56:54.026296 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ca6a37-5734-4798-9c68-1da4bf0a4572 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:54.027987 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ca6a37-5734-4798-9c68-1da4bf0a4572 service placement] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:54.039690 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1169/2329] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:56:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:56:54.070758 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f0d897ac-abc9-4f08-ae52-f90773724243 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6367371496f04cc2a7c105fbbed68dca'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:56:54.078697 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f0d897ac-abc9-4f08-ae52-f90773724243 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6367371496f04cc2a7c105fbbed68dca'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:56:54.083196 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:54.092244 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f0d897ac-abc9-4f08-ae52-f90773724243 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6367371496f04cc2a7c105fbbed68dca'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:56:54.093689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f0d897ac-abc9-4f08-ae52-f90773724243 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6367371496f04cc2a7c105fbbed68dca'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:56:54.094464 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f0d897ac-abc9-4f08-ae52-f90773724243 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6367371496f04cc2a7c105fbbed68dca'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:56:54.107062 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1161/2330] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:56:53 2022] DELETE /identity/v3/users/6367371496f04cc2a7c105fbbed68dca => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:56:54.286610 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca6a37-5734-4798-9c68-1da4bf0a4572 service placement] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:54.306277 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca6a37-5734-4798-9c68-1da4bf0a4572 service placement] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:54.367418 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ebe4d1-22b2-453d-ad4c-b0ad33f21674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:54.368051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebe4d1-22b2-453d-ad4c-b0ad33f21674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:54.368330 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebe4d1-22b2-453d-ad4c-b0ad33f21674 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:54.368602 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebe4d1-22b2-453d-ad4c-b0ad33f21674 admin admin] PATH_INFO: `/v3/projects/24b951d69396403b9f4b88cea1f3fa76` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:54.369743 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ebe4d1-22b2-453d-ad4c-b0ad33f21674 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24b951d69396403b9f4b88cea1f3fa76)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:54.371639 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ebe4d1-22b2-453d-ad4c-b0ad33f21674 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:54.419836 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-99ebe4d1-22b2-453d-ad4c-b0ad33f21674 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24b951d69396403b9f4b88cea1f3fa76', 'request_id': 'req-99ebe4d1-22b2-453d-ad4c-b0ad33f21674'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:56:54.425665 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-99ebe4d1-22b2-453d-ad4c-b0ad33f21674 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24b951d69396403b9f4b88cea1f3fa76', 'request_id': 'req-99ebe4d1-22b2-453d-ad4c-b0ad33f21674'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:56:54.434133 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-99ebe4d1-22b2-453d-ad4c-b0ad33f21674 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24b951d69396403b9f4b88cea1f3fa76', 'request_id': 'req-99ebe4d1-22b2-453d-ad4c-b0ad33f21674'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:56:54.445702 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-99ebe4d1-22b2-453d-ad4c-b0ad33f21674 admin admin] The token cache is being invalidate because project 24b951d69396403b9f4b88cea1f3fa76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:56:54.445994 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-99ebe4d1-22b2-453d-ad4c-b0ad33f21674 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:56:54.446680 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-99ebe4d1-22b2-453d-ad4c-b0ad33f21674 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:56:54.448403 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1170/2331] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:56:54 2022] DELETE /identity/v3/projects/24b951d69396403b9f4b88cea1f3fa76 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:56:54.454445 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4cf798-8405-46c9-a041-64bc2e85e752 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:54.454738 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4cf798-8405-46c9-a041-64bc2e85e752 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:54.455043 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4cf798-8405-46c9-a041-64bc2e85e752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:54.502368 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-ea4cf798-8405-46c9-a041-64bc2e85e752 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:56:54.503590 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-ea4cf798-8405-46c9-a041-64bc2e85e752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:56:54.569316 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1162/2332] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:56:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:56:54.578921 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fcd2c-7ba7-4797-bd9f-f540b10567ae None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:54.579207 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fcd2c-7ba7-4797-bd9f-f540b10567ae None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:54.579486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fcd2c-7ba7-4797-bd9f-f540b10567ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:54.599840 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-985fcd2c-7ba7-4797-bd9f-f540b10567ae None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:56:54.601056 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-985fcd2c-7ba7-4797-bd9f-f540b10567ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:56:54.609080 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1171/2333] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:56:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:56:54.613330 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4cf798-8405-46c9-a041-64bc2e85e752 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:54.623154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4cf798-8405-46c9-a041-64bc2e85e752 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:54.627839 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b4f0d-3476-48b6-b37d-bc26b3e34372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:54.628436 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b4f0d-3476-48b6-b37d-bc26b3e34372 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:54.628714 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b4f0d-3476-48b6-b37d-bc26b3e34372 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:54.628985 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b4f0d-3476-48b6-b37d-bc26b3e34372 admin admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:54.629665 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b4f0d-3476-48b6-b37d-bc26b3e34372 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:56:54.629967 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b4f0d-3476-48b6-b37d-bc26b3e34372 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:54.631559 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b4f0d-3476-48b6-b37d-bc26b3e34372 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:54.643870 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1163/2334] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:56:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:56:54.648051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985fcd2c-7ba7-4797-bd9f-f540b10567ae None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:54.658234 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985fcd2c-7ba7-4797-bd9f-f540b10567ae None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:54.663166 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9eb24e-cc6a-4ed2-9721-23e512a1766c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:54.663768 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9eb24e-cc6a-4ed2-9721-23e512a1766c admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:54.664050 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9eb24e-cc6a-4ed2-9721-23e512a1766c admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:54.664323 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9eb24e-cc6a-4ed2-9721-23e512a1766c admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:54.664962 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9eb24e-cc6a-4ed2-9721-23e512a1766c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:54.666961 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9eb24e-cc6a-4ed2-9721-23e512a1766c admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:54.679493 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:54.696461 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1172/2335] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:56:54 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:56:54.700700 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b4f0d-3476-48b6-b37d-bc26b3e34372 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:54.711537 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b4f0d-3476-48b6-b37d-bc26b3e34372 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:54.768559 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf4c213-28ea-4779-bbdc-2542ba33325e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:54.769158 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4c213-28ea-4779-bbdc-2542ba33325e admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:54.769467 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4c213-28ea-4779-bbdc-2542ba33325e admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:54.769736 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4c213-28ea-4779-bbdc-2542ba33325e admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:54.770354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf4c213-28ea-4779-bbdc-2542ba33325e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:54.772255 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf4c213-28ea-4779-bbdc-2542ba33325e admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:54.793543 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1164/2336] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:56:54 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:56:54.798352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9eb24e-cc6a-4ed2-9721-23e512a1766c admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:54.808665 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9eb24e-cc6a-4ed2-9721-23e512a1766c admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:54.813569 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70bea9-8b3d-4d3a-9c4d-56cdf619e0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:54.814172 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70bea9-8b3d-4d3a-9c4d-56cdf619e0a6 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:54.814458 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70bea9-8b3d-4d3a-9c4d-56cdf619e0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:54.814736 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70bea9-8b3d-4d3a-9c4d-56cdf619e0a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:54.815434 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70bea9-8b3d-4d3a-9c4d-56cdf619e0a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:54.817010 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70bea9-8b3d-4d3a-9c4d-56cdf619e0a6 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:54.826864 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1173/2337] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 22:56:54.831040 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf4c213-28ea-4779-bbdc-2542ba33325e admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:54.840928 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf4c213-28ea-4779-bbdc-2542ba33325e admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:54.846127 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730f8abc-086f-450f-84d9-c06534d5176c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:54.846722 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f8abc-086f-450f-84d9-c06534d5176c admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:54.846996 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f8abc-086f-450f-84d9-c06534d5176c admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:54.847266 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f8abc-086f-450f-84d9-c06534d5176c admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:54.847899 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730f8abc-086f-450f-84d9-c06534d5176c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:54.849522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730f8abc-086f-450f-84d9-c06534d5176c admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:54.860749 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1165/2338] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:56:54.864919 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70bea9-8b3d-4d3a-9c4d-56cdf619e0a6 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:54.877776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70bea9-8b3d-4d3a-9c4d-56cdf619e0a6 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:54.883461 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a6b279-39b0-4d34-8467-5550f95cc5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:54.884115 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6b279-39b0-4d34-8467-5550f95cc5c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:54.884393 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6b279-39b0-4d34-8467-5550f95cc5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:54.884691 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6b279-39b0-4d34-8467-5550f95cc5c4 admin admin] PATH_INFO: `/v3/projects/179da2fa522b41938284bd957ca28306/users/75d335503b944581acf82abadda5c7d7/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:54.901133 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a6b279-39b0-4d34-8467-5550f95cc5c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=179da2fa522b41938284bd957ca28306, user_id=75d335503b944581acf82abadda5c7d7, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:54.903926 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a6b279-39b0-4d34-8467-5550f95cc5c4 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:54.917379 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1174/2339] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:56:54 2022] PUT /identity/v3/projects/179da2fa522b41938284bd957ca28306/users/75d335503b944581acf82abadda5c7d7/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:56:54.928761 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8730a1-444b-4886-8972-145807161119 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:54.929041 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8730a1-444b-4886-8972-145807161119 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:54.929334 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8730a1-444b-4886-8972-145807161119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:54.956394 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-8d8730a1-444b-4886-8972-145807161119 None None] MFA Rules not processed for user `75d335503b944581acf82abadda5c7d7`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:56:54.957669 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-8d8730a1-444b-4886-8972-145807161119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:56:54.966786 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:55.024736 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1166/2340] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:56:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:56:55.029237 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a6b279-39b0-4d34-8467-5550f95cc5c4 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:55.091270 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a6b279-39b0-4d34-8467-5550f95cc5c4 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:55.096084 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c371a9c-a3ca-4742-b572-51ee73afdff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:55.096684 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c371a9c-a3ca-4742-b572-51ee73afdff1 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:55.096961 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c371a9c-a3ca-4742-b572-51ee73afdff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:55.097229 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c371a9c-a3ca-4742-b572-51ee73afdff1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:55.097884 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c371a9c-a3ca-4742-b572-51ee73afdff1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:55.099789 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c371a9c-a3ca-4742-b572-51ee73afdff1 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:55.128308 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1175/2341] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:56:55 2022] POST /identity/v3/projects => generated 365 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:56:55.132332 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8730a1-444b-4886-8972-145807161119 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:55.144498 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8730a1-444b-4886-8972-145807161119 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:55.217666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc9ad2-e91b-48e1-9340-b72685017ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:55.218533 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc9ad2-e91b-48e1-9340-b72685017ee6 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:55.218903 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc9ad2-e91b-48e1-9340-b72685017ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:55.219190 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc9ad2-e91b-48e1-9340-b72685017ee6 admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:55.219818 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc9ad2-e91b-48e1-9340-b72685017ee6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:55.221906 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc9ad2-e91b-48e1-9340-b72685017ee6 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:55.250993 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1167/2342] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:56:55 2022] POST /identity/v3/users => generated 289 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:56:55.255036 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c371a9c-a3ca-4742-b572-51ee73afdff1 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:55.265628 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c371a9c-a3ca-4742-b572-51ee73afdff1 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:55.270847 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4e1f4-1173-4a42-8d08-d77d25f5a02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:55.271507 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4e1f4-1173-4a42-8d08-d77d25f5a02b admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:55.271824 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4e1f4-1173-4a42-8d08-d77d25f5a02b admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:55.272128 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4e1f4-1173-4a42-8d08-d77d25f5a02b admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:55.272820 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a4e1f4-1173-4a42-8d08-d77d25f5a02b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:55.274490 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a4e1f4-1173-4a42-8d08-d77d25f5a02b admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:55.284928 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1176/2343] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:56:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:56:55.289125 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc9ad2-e91b-48e1-9340-b72685017ee6 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:55.299703 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc9ad2-e91b-48e1-9340-b72685017ee6 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:55.304301 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55a360-108a-431f-b2a5-209100b3fee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:55.304939 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55a360-108a-431f-b2a5-209100b3fee0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:55.305215 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55a360-108a-431f-b2a5-209100b3fee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:55.305519 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55a360-108a-431f-b2a5-209100b3fee0 admin admin] PATH_INFO: `/v3/projects/f169344f15194f4983cddf99e2446bcd/users/8c83d5b5be6e46b38b3cebb8d40cfc8c/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:55.321166 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd55a360-108a-431f-b2a5-209100b3fee0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f169344f15194f4983cddf99e2446bcd, user_id=8c83d5b5be6e46b38b3cebb8d40cfc8c, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:55.323930 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd55a360-108a-431f-b2a5-209100b3fee0 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:55.337641 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1168/2344] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:56:55 2022] PUT /identity/v3/projects/f169344f15194f4983cddf99e2446bcd/users/8c83d5b5be6e46b38b3cebb8d40cfc8c/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:56:55.341584 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4e1f4-1173-4a42-8d08-d77d25f5a02b admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:55.351471 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4e1f4-1173-4a42-8d08-d77d25f5a02b admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:55.407924 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ab220d-ef26-4601-96fe-de3d273b4de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:55.408451 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab220d-ef26-4601-96fe-de3d273b4de1 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:55.408840 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab220d-ef26-4601-96fe-de3d273b4de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:55.409113 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab220d-ef26-4601-96fe-de3d273b4de1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:55.409794 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ab220d-ef26-4601-96fe-de3d273b4de1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:55.411389 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ab220d-ef26-4601-96fe-de3d273b4de1 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:55.423428 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1177/2345] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:56:55 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:56:55.427675 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55a360-108a-431f-b2a5-209100b3fee0 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:55.438687 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55a360-108a-431f-b2a5-209100b3fee0 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:55.443217 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2db4a-6237-4c90-b936-fa2015f8e315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:55.443862 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2db4a-6237-4c90-b936-fa2015f8e315 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:55.444138 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2db4a-6237-4c90-b936-fa2015f8e315 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:55.444406 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2db4a-6237-4c90-b936-fa2015f8e315 admin admin] PATH_INFO: `/v3/projects/f169344f15194f4983cddf99e2446bcd/users/8c83d5b5be6e46b38b3cebb8d40cfc8c/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:55.446338 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f2db4a-6237-4c90-b936-fa2015f8e315 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f169344f15194f4983cddf99e2446bcd, user_id=8c83d5b5be6e46b38b3cebb8d40cfc8c, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:55.448962 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f2db4a-6237-4c90-b936-fa2015f8e315 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:55.459051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1169/2346] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:56:55 2022] PUT /identity/v3/projects/f169344f15194f4983cddf99e2446bcd/users/8c83d5b5be6e46b38b3cebb8d40cfc8c/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:56:55.470645 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44235439-6c33-47ba-a093-5e0e7c0eb2d1 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:55.470931 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44235439-6c33-47ba-a093-5e0e7c0eb2d1 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:55.471210 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44235439-6c33-47ba-a093-5e0e7c0eb2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:55.498585 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-44235439-6c33-47ba-a093-5e0e7c0eb2d1 None None] MFA Rules not processed for user `8c83d5b5be6e46b38b3cebb8d40cfc8c`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:56:55.500483 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-44235439-6c33-47ba-a093-5e0e7c0eb2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:56:55.513237 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:55.572493 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1178/2347] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:56:55 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:56:55.583965 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2db4a-6237-4c90-b936-fa2015f8e315 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:55.585709 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-f9f2db4a-6237-4c90-b936-fa2015f8e315 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:56:55.662669 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:56:55.671879 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2db4a-6237-4c90-b936-fa2015f8e315 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:55.675530 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04985daa-b26c-4342-9484-76b1d28ed1b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:55.676127 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04985daa-b26c-4342-9484-76b1d28ed1b0 service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:55.676401 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04985daa-b26c-4342-9484-76b1d28ed1b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:55.676670 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04985daa-b26c-4342-9484-76b1d28ed1b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:55.685527 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04985daa-b26c-4342-9484-76b1d28ed1b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:55.687402 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04985daa-b26c-4342-9484-76b1d28ed1b0 service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:55.697581 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1170/2348] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:56:55 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:56:55.773853 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44235439-6c33-47ba-a093-5e0e7c0eb2d1 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:56:55.784457 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44235439-6c33-47ba-a093-5e0e7c0eb2d1 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:56:55.788099 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0629e9-461d-4444-bd54-ef34961f3da0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:56:55.788711 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0629e9-461d-4444-bd54-ef34961f3da0 service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:56:55.788988 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0629e9-461d-4444-bd54-ef34961f3da0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:56:55.789256 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0629e9-461d-4444-bd54-ef34961f3da0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:56:55.851845 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0629e9-461d-4444-bd54-ef34961f3da0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:56:55.853418 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0629e9-461d-4444-bd54-ef34961f3da0 service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:56:55.864177 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1179/2349] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:56:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:57:05.692791 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=6 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:57:05.859481 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=50 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:57:07.209914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04985daa-b26c-4342-9484-76b1d28ed1b0 service cinder] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:07.272930 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04985daa-b26c-4342-9484-76b1d28ed1b0 service cinder] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:07.277929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0202018-706d-470f-952d-6898e6bba651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:07.278487 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0202018-706d-470f-952d-6898e6bba651 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:07.278766 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0202018-706d-470f-952d-6898e6bba651 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:07.279035 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0202018-706d-470f-952d-6898e6bba651 admin admin] PATH_INFO: `/v3/users/75d335503b944581acf82abadda5c7d7` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:07.280773 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0202018-706d-470f-952d-6898e6bba651 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75d335503b944581acf82abadda5c7d7)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:07.282807 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0202018-706d-470f-952d-6898e6bba651 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:07.332546 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e0202018-706d-470f-952d-6898e6bba651 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75d335503b944581acf82abadda5c7d7'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:07.339451 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e0202018-706d-470f-952d-6898e6bba651 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75d335503b944581acf82abadda5c7d7'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:07.351337 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e0202018-706d-470f-952d-6898e6bba651 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75d335503b944581acf82abadda5c7d7'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:07.352080 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e0202018-706d-470f-952d-6898e6bba651 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75d335503b944581acf82abadda5c7d7'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:07.352692 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e0202018-706d-470f-952d-6898e6bba651 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75d335503b944581acf82abadda5c7d7'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:07.367013 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1171/2350] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:57:07 2022] DELETE /identity/v3/users/75d335503b944581acf82abadda5c7d7 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:57:07.371749 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7faa2-4bbb-45f3-9d02-c3c96cd9c348 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:07.372234 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7faa2-4bbb-45f3-9d02-c3c96cd9c348 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:07.372335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7faa2-4bbb-45f3-9d02-c3c96cd9c348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:07.391985 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-fae7faa2-4bbb-45f3-9d02-c3c96cd9c348 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:57:07.393194 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-fae7faa2-4bbb-45f3-9d02-c3c96cd9c348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:07.451688 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1180/2351] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:57:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:57:07.461859 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0202018-706d-470f-952d-6898e6bba651 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:07.463271 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-e0202018-706d-470f-952d-6898e6bba651 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:07.520388 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0202018-706d-470f-952d-6898e6bba651 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:07.524088 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b2c4c-113a-4479-90fc-06b7882be0d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:07.524687 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b2c4c-113a-4479-90fc-06b7882be0d9 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:07.524966 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b2c4c-113a-4479-90fc-06b7882be0d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:07.525238 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b2c4c-113a-4479-90fc-06b7882be0d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:07.534909 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5b2c4c-113a-4479-90fc-06b7882be0d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:07.536495 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5b2c4c-113a-4479-90fc-06b7882be0d9 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:07.545819 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1172/2352] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:57:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:57:07.813175 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae7faa2-4bbb-45f3-9d02-c3c96cd9c348 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:07.814773 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-fae7faa2-4bbb-45f3-9d02-c3c96cd9c348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:07.829121 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae7faa2-4bbb-45f3-9d02-c3c96cd9c348 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:07.833864 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc0b11-ab31-4169-9b92-d6d65113cbb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:07.834747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc0b11-ab31-4169-9b92-d6d65113cbb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:07.834747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc0b11-ab31-4169-9b92-d6d65113cbb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:07.835039 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc0b11-ab31-4169-9b92-d6d65113cbb1 admin admin] PATH_INFO: `/v3/projects/179da2fa522b41938284bd957ca28306` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:07.836205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bc0b11-ab31-4169-9b92-d6d65113cbb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=179da2fa522b41938284bd957ca28306)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:07.838239 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bc0b11-ab31-4169-9b92-d6d65113cbb1 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:07.885381 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-c9bc0b11-ab31-4169-9b92-d6d65113cbb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '179da2fa522b41938284bd957ca28306', 'request_id': 'req-c9bc0b11-ab31-4169-9b92-d6d65113cbb1'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:07.891721 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-c9bc0b11-ab31-4169-9b92-d6d65113cbb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '179da2fa522b41938284bd957ca28306', 'request_id': 'req-c9bc0b11-ab31-4169-9b92-d6d65113cbb1'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:07.899487 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-c9bc0b11-ab31-4169-9b92-d6d65113cbb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '179da2fa522b41938284bd957ca28306', 'request_id': 'req-c9bc0b11-ab31-4169-9b92-d6d65113cbb1'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:07.911979 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-c9bc0b11-ab31-4169-9b92-d6d65113cbb1 admin admin] The token cache is being invalidate because project 179da2fa522b41938284bd957ca28306 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:57:07.912272 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-c9bc0b11-ab31-4169-9b92-d6d65113cbb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:07.912902 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-c9bc0b11-ab31-4169-9b92-d6d65113cbb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:07.914400 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1181/2353] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:57:07 2022] DELETE /identity/v3/projects/179da2fa522b41938284bd957ca28306 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:57:07.918366 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b2c4c-113a-4479-90fc-06b7882be0d9 service neutron] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:07.928238 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b2c4c-113a-4479-90fc-06b7882be0d9 service neutron] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:07.947351 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:57:07.989492 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb28c28-7db4-4395-8891-0a4b66034d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:07.990132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb28c28-7db4-4395-8891-0a4b66034d85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:07.990414 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb28c28-7db4-4395-8891-0a4b66034d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:07.990692 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb28c28-7db4-4395-8891-0a4b66034d85 admin admin] PATH_INFO: `/v3/users/8c83d5b5be6e46b38b3cebb8d40cfc8c` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:07.991819 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb28c28-7db4-4395-8891-0a4b66034d85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c83d5b5be6e46b38b3cebb8d40cfc8c)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:07.993795 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb28c28-7db4-4395-8891-0a4b66034d85 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:08.043531 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-0bb28c28-7db4-4395-8891-0a4b66034d85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c83d5b5be6e46b38b3cebb8d40cfc8c'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:08.051776 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-0bb28c28-7db4-4395-8891-0a4b66034d85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c83d5b5be6e46b38b3cebb8d40cfc8c'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:08.065543 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-0bb28c28-7db4-4395-8891-0a4b66034d85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c83d5b5be6e46b38b3cebb8d40cfc8c'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:08.066336 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-0bb28c28-7db4-4395-8891-0a4b66034d85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c83d5b5be6e46b38b3cebb8d40cfc8c'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:08.067084 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-0bb28c28-7db4-4395-8891-0a4b66034d85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c83d5b5be6e46b38b3cebb8d40cfc8c'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:08.080845 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1173/2354] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:57:07 2022] DELETE /identity/v3/users/8c83d5b5be6e46b38b3cebb8d40cfc8c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:57:08.354434 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc0b11-ab31-4169-9b92-d6d65113cbb1 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:08.355970 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-c9bc0b11-ab31-4169-9b92-d6d65113cbb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:08.369708 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc0b11-ab31-4169-9b92-d6d65113cbb1 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:08.426533 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec083b4-261e-4c95-b2eb-3cd56752e052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:08.427190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec083b4-261e-4c95-b2eb-3cd56752e052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:08.427440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec083b4-261e-4c95-b2eb-3cd56752e052 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:08.427713 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec083b4-261e-4c95-b2eb-3cd56752e052 admin admin] PATH_INFO: `/v3/projects/f169344f15194f4983cddf99e2446bcd` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:08.428840 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec083b4-261e-4c95-b2eb-3cd56752e052 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f169344f15194f4983cddf99e2446bcd)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:08.430881 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec083b4-261e-4c95-b2eb-3cd56752e052 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:08.462365 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:57:08.479768 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-fec083b4-261e-4c95-b2eb-3cd56752e052 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f169344f15194f4983cddf99e2446bcd', 'request_id': 'req-fec083b4-261e-4c95-b2eb-3cd56752e052'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:08.485647 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-fec083b4-261e-4c95-b2eb-3cd56752e052 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f169344f15194f4983cddf99e2446bcd', 'request_id': 'req-fec083b4-261e-4c95-b2eb-3cd56752e052'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:08.492852 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-fec083b4-261e-4c95-b2eb-3cd56752e052 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f169344f15194f4983cddf99e2446bcd', 'request_id': 'req-fec083b4-261e-4c95-b2eb-3cd56752e052'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:08.504679 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-fec083b4-261e-4c95-b2eb-3cd56752e052 admin admin] The token cache is being invalidate because project f169344f15194f4983cddf99e2446bcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:57:08.504969 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-fec083b4-261e-4c95-b2eb-3cd56752e052 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:08.505617 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-fec083b4-261e-4c95-b2eb-3cd56752e052 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:08.507092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1182/2355] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:57:08 2022] DELETE /identity/v3/projects/f169344f15194f4983cddf99e2446bcd => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:57:08.512886 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ce648-cd88-4db7-b42e-55b530223886 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:08.513399 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ce648-cd88-4db7-b42e-55b530223886 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:08.513498 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ce648-cd88-4db7-b42e-55b530223886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:08.533516 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-5b6ce648-cd88-4db7-b42e-55b530223886 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:57:08.534724 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-5b6ce648-cd88-4db7-b42e-55b530223886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:08.594419 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1174/2356] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:57:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:57:08.602201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62735c52-aa78-4053-baa1-48646cfcdd1e None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:08.602426 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62735c52-aa78-4053-baa1-48646cfcdd1e None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:08.602732 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62735c52-aa78-4053-baa1-48646cfcdd1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:08.623601 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-62735c52-aa78-4053-baa1-48646cfcdd1e None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:57:08.625643 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-62735c52-aa78-4053-baa1-48646cfcdd1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:08.633531 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1183/2357] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:57:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:57:08.637955 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ce648-cd88-4db7-b42e-55b530223886 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:08.648439 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ce648-cd88-4db7-b42e-55b530223886 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:08.653063 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5973bec9-692f-4c6c-a60b-a0843633f8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:08.653706 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973bec9-692f-4c6c-a60b-a0843633f8b2 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:08.653990 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973bec9-692f-4c6c-a60b-a0843633f8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:08.654265 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973bec9-692f-4c6c-a60b-a0843633f8b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:08.654909 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5973bec9-692f-4c6c-a60b-a0843633f8b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:57:08.655462 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5973bec9-692f-4c6c-a60b-a0843633f8b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:08.657350 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5973bec9-692f-4c6c-a60b-a0843633f8b2 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:08.668766 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1175/2358] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:57:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:57:08.673740 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62735c52-aa78-4053-baa1-48646cfcdd1e None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:08.682987 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62735c52-aa78-4053-baa1-48646cfcdd1e None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:08.688315 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d1009-7972-47f4-8eed-cab38b810701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:08.688899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d1009-7972-47f4-8eed-cab38b810701 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:08.689177 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d1009-7972-47f4-8eed-cab38b810701 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:08.689479 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d1009-7972-47f4-8eed-cab38b810701 admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:08.690105 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3d1009-7972-47f4-8eed-cab38b810701 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:08.691993 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3d1009-7972-47f4-8eed-cab38b810701 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:08.718064 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1184/2359] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:57:08 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:57:08.722252 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5973bec9-692f-4c6c-a60b-a0843633f8b2 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:08.732605 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5973bec9-692f-4c6c-a60b-a0843633f8b2 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:08.758579 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:57:08.790118 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d415bcdd-6e01-45b4-8dbd-b9972def476c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:08.790723 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d415bcdd-6e01-45b4-8dbd-b9972def476c admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:08.791000 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d415bcdd-6e01-45b4-8dbd-b9972def476c admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:08.791268 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d415bcdd-6e01-45b4-8dbd-b9972def476c admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:08.791905 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d415bcdd-6e01-45b4-8dbd-b9972def476c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:08.793862 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d415bcdd-6e01-45b4-8dbd-b9972def476c admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:08.816080 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1176/2360] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:57:08 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:57:08.820309 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d1009-7972-47f4-8eed-cab38b810701 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:08.831000 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d1009-7972-47f4-8eed-cab38b810701 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:08.835598 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b4afe8-07c5-4639-8aea-480e707573e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:08.836192 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b4afe8-07c5-4639-8aea-480e707573e3 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:08.836467 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b4afe8-07c5-4639-8aea-480e707573e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:08.836738 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b4afe8-07c5-4639-8aea-480e707573e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:08.837404 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b4afe8-07c5-4639-8aea-480e707573e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:08.839038 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b4afe8-07c5-4639-8aea-480e707573e3 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:08.849261 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1185/2361] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:57:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:57:08.853109 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d415bcdd-6e01-45b4-8dbd-b9972def476c admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:08.863454 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d415bcdd-6e01-45b4-8dbd-b9972def476c admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:08.868071 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4afe6a-c8b5-477e-88f2-469418f88cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:08.868659 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4afe6a-c8b5-477e-88f2-469418f88cbd admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:08.868933 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4afe6a-c8b5-477e-88f2-469418f88cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:08.869200 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4afe6a-c8b5-477e-88f2-469418f88cbd admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:08.869867 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4afe6a-c8b5-477e-88f2-469418f88cbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:08.871441 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4afe6a-c8b5-477e-88f2-469418f88cbd admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:08.883734 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1177/2362] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:57:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:57:08.888111 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b4afe8-07c5-4639-8aea-480e707573e3 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:08.899190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b4afe8-07c5-4639-8aea-480e707573e3 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:08.903849 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43111a-9dcd-49a8-aa09-feaa23031029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:08.904493 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43111a-9dcd-49a8-aa09-feaa23031029 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:08.904770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43111a-9dcd-49a8-aa09-feaa23031029 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:08.905076 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43111a-9dcd-49a8-aa09-feaa23031029 admin admin] PATH_INFO: `/v3/projects/bf31b51c6c4748d5883ad2ecea5a7924/users/a42047e0b3084a1e8bd417bcdbf8a0ce/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:08.919327 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43111a-9dcd-49a8-aa09-feaa23031029 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf31b51c6c4748d5883ad2ecea5a7924, user_id=a42047e0b3084a1e8bd417bcdbf8a0ce, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:08.922678 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43111a-9dcd-49a8-aa09-feaa23031029 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:08.935069 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1186/2363] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:57:08 2022] PUT /identity/v3/projects/bf31b51c6c4748d5883ad2ecea5a7924/users/a42047e0b3084a1e8bd417bcdbf8a0ce/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:57:08.945741 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37e5b5-7808-4257-9fe2-187c10260aa7 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:08.946050 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37e5b5-7808-4257-9fe2-187c10260aa7 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:08.946343 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37e5b5-7808-4257-9fe2-187c10260aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:08.972955 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-2d37e5b5-7808-4257-9fe2-187c10260aa7 None None] MFA Rules not processed for user `a42047e0b3084a1e8bd417bcdbf8a0ce`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:57:08.974233 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-2d37e5b5-7808-4257-9fe2-187c10260aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:09.037464 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1178/2364] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:57:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:57:09.042090 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43111a-9dcd-49a8-aa09-feaa23031029 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:09.103027 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43111a-9dcd-49a8-aa09-feaa23031029 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:09.107925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8da6df-ce98-42ce-a421-cc689d8d753c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:09.109786 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8da6df-ce98-42ce-a421-cc689d8d753c admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:09.109786 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8da6df-ce98-42ce-a421-cc689d8d753c admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:09.110154 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8da6df-ce98-42ce-a421-cc689d8d753c admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:09.110876 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8da6df-ce98-42ce-a421-cc689d8d753c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:09.112794 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8da6df-ce98-42ce-a421-cc689d8d753c admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:09.139856 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1187/2365] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:57:09 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:57:09.143841 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d37e5b5-7808-4257-9fe2-187c10260aa7 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:09.156475 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d37e5b5-7808-4257-9fe2-187c10260aa7 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:09.213938 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d45be5-7e00-4915-be2d-d1854bd004ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:09.214556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d45be5-7e00-4915-be2d-d1854bd004ec admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:09.214832 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d45be5-7e00-4915-be2d-d1854bd004ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:09.215100 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d45be5-7e00-4915-be2d-d1854bd004ec admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:09.215751 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d45be5-7e00-4915-be2d-d1854bd004ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:09.218420 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d45be5-7e00-4915-be2d-d1854bd004ec admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:09.252913 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1179/2366] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:57:09 2022] POST /identity/v3/users => generated 294 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:57:09.254230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8da6df-ce98-42ce-a421-cc689d8d753c admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:09.265400 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8da6df-ce98-42ce-a421-cc689d8d753c admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:09.270382 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ab7eb0-3160-42c2-9840-25a44623843c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:09.271014 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab7eb0-3160-42c2-9840-25a44623843c admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:09.271281 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab7eb0-3160-42c2-9840-25a44623843c admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:09.271556 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab7eb0-3160-42c2-9840-25a44623843c admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:09.272332 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ab7eb0-3160-42c2-9840-25a44623843c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:09.273974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ab7eb0-3160-42c2-9840-25a44623843c admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:09.286631 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1188/2367] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:57:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 22:57:09.290877 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d45be5-7e00-4915-be2d-d1854bd004ec admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:09.301935 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d45be5-7e00-4915-be2d-d1854bd004ec admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:09.306632 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7def37-e45e-4d72-8be5-fe0ff5592cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:09.307292 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7def37-e45e-4d72-8be5-fe0ff5592cf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:09.307573 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7def37-e45e-4d72-8be5-fe0ff5592cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:09.307850 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7def37-e45e-4d72-8be5-fe0ff5592cf8 admin admin] PATH_INFO: `/v3/projects/ada67ff872b54f72a0c5aa8ca570c5bf/users/ed8d715515a9427e8737413999aaf38f/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:09.322805 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7def37-e45e-4d72-8be5-fe0ff5592cf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ada67ff872b54f72a0c5aa8ca570c5bf, user_id=ed8d715515a9427e8737413999aaf38f, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:09.325517 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7def37-e45e-4d72-8be5-fe0ff5592cf8 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:09.338665 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1180/2368] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:57:09 2022] PUT /identity/v3/projects/ada67ff872b54f72a0c5aa8ca570c5bf/users/ed8d715515a9427e8737413999aaf38f/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:57:09.343258 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ab7eb0-3160-42c2-9840-25a44623843c admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:09.353306 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ab7eb0-3160-42c2-9840-25a44623843c admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:09.373247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:57:09.412788 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b66e009-125f-4879-86c1-9beeb1b6513d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:09.413434 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b66e009-125f-4879-86c1-9beeb1b6513d admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:09.413714 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b66e009-125f-4879-86c1-9beeb1b6513d admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:09.414105 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b66e009-125f-4879-86c1-9beeb1b6513d admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:09.414742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b66e009-125f-4879-86c1-9beeb1b6513d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:09.416338 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b66e009-125f-4879-86c1-9beeb1b6513d admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:09.426425 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1189/2369] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:57:09 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:57:09.430843 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7def37-e45e-4d72-8be5-fe0ff5592cf8 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:09.436328 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:57:09.442636 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7def37-e45e-4d72-8be5-fe0ff5592cf8 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:09.447456 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2415968-bf36-468b-9cf1-7dc9a295b64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:09.448098 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2415968-bf36-468b-9cf1-7dc9a295b64f admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:09.448376 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2415968-bf36-468b-9cf1-7dc9a295b64f admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:09.448647 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2415968-bf36-468b-9cf1-7dc9a295b64f admin admin] PATH_INFO: `/v3/projects/ada67ff872b54f72a0c5aa8ca570c5bf/users/ed8d715515a9427e8737413999aaf38f/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:09.450545 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2415968-bf36-468b-9cf1-7dc9a295b64f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ada67ff872b54f72a0c5aa8ca570c5bf, user_id=ed8d715515a9427e8737413999aaf38f, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:09.453801 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2415968-bf36-468b-9cf1-7dc9a295b64f admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:09.463813 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1181/2370] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:57:09 2022] PUT /identity/v3/projects/ada67ff872b54f72a0c5aa8ca570c5bf/users/ed8d715515a9427e8737413999aaf38f/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:57:09.475920 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e65dbd-0384-4709-806b-cecc8fb9d714 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:09.476204 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e65dbd-0384-4709-806b-cecc8fb9d714 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:09.476479 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e65dbd-0384-4709-806b-cecc8fb9d714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:09.505965 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-95e65dbd-0384-4709-806b-cecc8fb9d714 None None] MFA Rules not processed for user `ed8d715515a9427e8737413999aaf38f`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:57:09.507186 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-95e65dbd-0384-4709-806b-cecc8fb9d714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:09.580231 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1190/2371] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:57:09 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:57:09.591443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2415968-bf36-468b-9cf1-7dc9a295b64f admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:09.593961 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-b2415968-bf36-468b-9cf1-7dc9a295b64f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:09.651432 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2415968-bf36-468b-9cf1-7dc9a295b64f admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:09.655829 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c064126-1400-42f6-8651-739022f94c22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:09.656562 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c064126-1400-42f6-8651-739022f94c22 service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:09.656843 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c064126-1400-42f6-8651-739022f94c22 service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:09.657114 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c064126-1400-42f6-8651-739022f94c22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:09.712689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c064126-1400-42f6-8651-739022f94c22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:09.714178 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c064126-1400-42f6-8651-739022f94c22 service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:09.724002 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1182/2372] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:57:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:57:16.406281 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e65dbd-0384-4709-806b-cecc8fb9d714 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:16.416365 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e65dbd-0384-4709-806b-cecc8fb9d714 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:16.419963 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a48b6a-4d0b-413c-a9c7-d1c6fc99b8c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:16.420671 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a48b6a-4d0b-413c-a9c7-d1c6fc99b8c9 service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:16.420948 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a48b6a-4d0b-413c-a9c7-d1c6fc99b8c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:16.421215 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a48b6a-4d0b-413c-a9c7-d1c6fc99b8c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:16.431760 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a48b6a-4d0b-413c-a9c7-d1c6fc99b8c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:16.433204 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a48b6a-4d0b-413c-a9c7-d1c6fc99b8c9 service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:16.443709 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1191/2373] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:57:16 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:57:19.719940 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=52,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:57:22.476036 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c064126-1400-42f6-8651-739022f94c22 service cinder] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:22.477880 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-6c064126-1400-42f6-8651-739022f94c22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:22.489547 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c064126-1400-42f6-8651-739022f94c22 service cinder] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:22.493851 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd161850-2496-438f-a7ee-a4e90ec76eed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:22.494523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd161850-2496-438f-a7ee-a4e90ec76eed service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:22.494809 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd161850-2496-438f-a7ee-a4e90ec76eed service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:22.495084 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd161850-2496-438f-a7ee-a4e90ec76eed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:22.496932 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-fd161850-2496-438f-a7ee-a4e90ec76eed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:22.579315 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd161850-2496-438f-a7ee-a4e90ec76eed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:22.580919 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd161850-2496-438f-a7ee-a4e90ec76eed service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:22.592506 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1183/2374] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:57:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:57:25.663826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a48b6a-4d0b-413c-a9c7-d1c6fc99b8c9 service cinder] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:25.665388 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-88a48b6a-4d0b-413c-a9c7-d1c6fc99b8c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:25.722811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a48b6a-4d0b-413c-a9c7-d1c6fc99b8c9 service cinder] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:25.726471 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe45a03-ca44-4594-933a-d3e59b57fda9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:25.727077 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe45a03-ca44-4594-933a-d3e59b57fda9 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:25.727359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe45a03-ca44-4594-933a-d3e59b57fda9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:25.727633 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe45a03-ca44-4594-933a-d3e59b57fda9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:25.729431 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-1fe45a03-ca44-4594-933a-d3e59b57fda9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:25.743473 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe45a03-ca44-4594-933a-d3e59b57fda9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:25.744895 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe45a03-ca44-4594-933a-d3e59b57fda9 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:25.755833 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1192/2375] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:57:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:57:32.587303 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=34 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:57:33.967435 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd161850-2496-438f-a7ee-a4e90ec76eed service cinder] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:33.968977 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-fd161850-2496-438f-a7ee-a4e90ec76eed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:33.983112 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd161850-2496-438f-a7ee-a4e90ec76eed service cinder] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:33.988767 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b75bf39-b394-4b43-b06e-a6eb6ec704eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:33.989475 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75bf39-b394-4b43-b06e-a6eb6ec704eb service nova] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:33.989759 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75bf39-b394-4b43-b06e-a6eb6ec704eb service nova] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:33.990035 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75bf39-b394-4b43-b06e-a6eb6ec704eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:33.991770 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-2b75bf39-b394-4b43-b06e-a6eb6ec704eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:34.074910 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b75bf39-b394-4b43-b06e-a6eb6ec704eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:34.076506 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b75bf39-b394-4b43-b06e-a6eb6ec704eb service nova] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:34.096300 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1184/2376] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:57:33 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:57:35.751196 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=84 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:57:41.845756 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe45a03-ca44-4594-933a-d3e59b57fda9 service neutron] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:41.847211 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-1fe45a03-ca44-4594-933a-d3e59b57fda9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:41.861241 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe45a03-ca44-4594-933a-d3e59b57fda9 service neutron] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:41.866776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9078c9-c8b6-4b67-8e8b-ac78300d1a0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:41.867368 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9078c9-c8b6-4b67-8e8b-ac78300d1a0f service nova] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:41.867646 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9078c9-c8b6-4b67-8e8b-ac78300d1a0f service nova] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:41.867916 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9078c9-c8b6-4b67-8e8b-ac78300d1a0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:41.869626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-5a9078c9-c8b6-4b67-8e8b-ac78300d1a0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:41.948089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9078c9-c8b6-4b67-8e8b-ac78300d1a0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:41.949718 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9078c9-c8b6-4b67-8e8b-ac78300d1a0f service nova] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:41.977233 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1193/2377] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:57:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:57:44.092913 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=41 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:57:47.162280 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d94094-c131-4f51-9a6e-a80b4c830978 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:47.164006 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d94094-c131-4f51-9a6e-a80b4c830978 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:47.164538 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d94094-c131-4f51-9a6e-a80b4c830978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:47.193531 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-45d94094-c131-4f51-9a6e-a80b4c830978 None None] MFA Rules not processed for user `aff5e804c64345b298c745287a1d0a32`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:57:47.195946 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-45d94094-c131-4f51-9a6e-a80b4c830978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:47.303982 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1185/2378] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 10 22:57:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:57:47.378865 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9078c9-c8b6-4b67-8e8b-ac78300d1a0f service nova] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:47.380894 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-5a9078c9-c8b6-4b67-8e8b-ac78300d1a0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:47.473795 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9078c9-c8b6-4b67-8e8b-ac78300d1a0f service nova] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:47.479350 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64769be-eb5a-4d80-8ab9-cb0becbe4ea3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e3a76df95a34b9fb9816f22b85d1071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:47.480172 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64769be-eb5a-4d80-8ab9-cb0becbe4ea3 service swift] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:47.480530 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64769be-eb5a-4d80-8ab9-cb0becbe4ea3 service swift] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:47.480910 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64769be-eb5a-4d80-8ab9-cb0becbe4ea3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:47.491186 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64769be-eb5a-4d80-8ab9-cb0becbe4ea3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:47.492856 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64769be-eb5a-4d80-8ab9-cb0becbe4ea3 service swift] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:47.502472 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1194/2379] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 10 22:57:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 22:57:49.821357 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d94094-c131-4f51-9a6e-a80b4c830978 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:49.924985 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d94094-c131-4f51-9a6e-a80b4c830978 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:49.931266 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f033b-48d2-47be-9af8-2a9ef48a4001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:49.932321 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f033b-48d2-47be-9af8-2a9ef48a4001 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:49.932773 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f033b-48d2-47be-9af8-2a9ef48a4001 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:49.933219 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f033b-48d2-47be-9af8-2a9ef48a4001 admin admin] PATH_INFO: `/v3/users/a42047e0b3084a1e8bd417bcdbf8a0ce` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:49.934918 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3f033b-48d2-47be-9af8-2a9ef48a4001 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a42047e0b3084a1e8bd417bcdbf8a0ce)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:49.937451 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3f033b-48d2-47be-9af8-2a9ef48a4001 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:50.002381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-de3f033b-48d2-47be-9af8-2a9ef48a4001 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a42047e0b3084a1e8bd417bcdbf8a0ce'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:50.017231 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-de3f033b-48d2-47be-9af8-2a9ef48a4001 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a42047e0b3084a1e8bd417bcdbf8a0ce'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:50.040529 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-de3f033b-48d2-47be-9af8-2a9ef48a4001 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a42047e0b3084a1e8bd417bcdbf8a0ce'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:50.041948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-de3f033b-48d2-47be-9af8-2a9ef48a4001 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a42047e0b3084a1e8bd417bcdbf8a0ce'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:50.042992 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-de3f033b-48d2-47be-9af8-2a9ef48a4001 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a42047e0b3084a1e8bd417bcdbf8a0ce'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:50.073810 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1186/2380] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:57:49 2022] DELETE /identity/v3/users/a42047e0b3084a1e8bd417bcdbf8a0ce => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:57:50.075874 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af15aa-4d61-4bac-8360-63d10ccd36bf None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:50.079287 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af15aa-4d61-4bac-8360-63d10ccd36bf None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:50.079817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af15aa-4d61-4bac-8360-63d10ccd36bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:50.136777 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-97af15aa-4d61-4bac-8360-63d10ccd36bf None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:57:50.138118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-97af15aa-4d61-4bac-8360-63d10ccd36bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:50.232155 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:57:50.245354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1195/2381] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:57:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:57:50.254636 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f033b-48d2-47be-9af8-2a9ef48a4001 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:50.256427 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-de3f033b-48d2-47be-9af8-2a9ef48a4001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:50.301177 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:57:50.348567 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f033b-48d2-47be-9af8-2a9ef48a4001 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:50.352574 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be5297-62d5-4f2d-8db8-d75ef8c66922 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:50.353186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be5297-62d5-4f2d-8db8-d75ef8c66922 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:50.353504 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be5297-62d5-4f2d-8db8-d75ef8c66922 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:50.353774 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be5297-62d5-4f2d-8db8-d75ef8c66922 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:50.363647 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54be5297-62d5-4f2d-8db8-d75ef8c66922 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:50.365088 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54be5297-62d5-4f2d-8db8-d75ef8c66922 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:50.377441 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1187/2382] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:57:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:57:50.793700 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97af15aa-4d61-4bac-8360-63d10ccd36bf None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:50.796078 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-97af15aa-4d61-4bac-8360-63d10ccd36bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:50.812097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97af15aa-4d61-4bac-8360-63d10ccd36bf None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:50.821612 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8659e5bc-73c6-46d6-b08b-e1ec1dca29cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:50.822744 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659e5bc-73c6-46d6-b08b-e1ec1dca29cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:50.823095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659e5bc-73c6-46d6-b08b-e1ec1dca29cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:50.823466 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659e5bc-73c6-46d6-b08b-e1ec1dca29cf admin admin] PATH_INFO: `/v3/projects/bf31b51c6c4748d5883ad2ecea5a7924` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:50.825013 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8659e5bc-73c6-46d6-b08b-e1ec1dca29cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf31b51c6c4748d5883ad2ecea5a7924)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:50.829622 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8659e5bc-73c6-46d6-b08b-e1ec1dca29cf admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:50.921484 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-8659e5bc-73c6-46d6-b08b-e1ec1dca29cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf31b51c6c4748d5883ad2ecea5a7924', 'request_id': 'req-8659e5bc-73c6-46d6-b08b-e1ec1dca29cf'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:50.946106 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-8659e5bc-73c6-46d6-b08b-e1ec1dca29cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf31b51c6c4748d5883ad2ecea5a7924', 'request_id': 'req-8659e5bc-73c6-46d6-b08b-e1ec1dca29cf'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:51.008894 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-8659e5bc-73c6-46d6-b08b-e1ec1dca29cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf31b51c6c4748d5883ad2ecea5a7924', 'request_id': 'req-8659e5bc-73c6-46d6-b08b-e1ec1dca29cf'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:51.027450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-8659e5bc-73c6-46d6-b08b-e1ec1dca29cf admin admin] The token cache is being invalidate because project bf31b51c6c4748d5883ad2ecea5a7924 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:57:51.027744 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-8659e5bc-73c6-46d6-b08b-e1ec1dca29cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:51.028426 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-8659e5bc-73c6-46d6-b08b-e1ec1dca29cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:51.030710 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1196/2383] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:57:50 2022] DELETE /identity/v3/projects/bf31b51c6c4748d5883ad2ecea5a7924 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:57:51.035745 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be5297-62d5-4f2d-8db8-d75ef8c66922 service neutron] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:51.047457 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be5297-62d5-4f2d-8db8-d75ef8c66922 service neutron] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:51.147940 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f820a-f7ad-4fbb-a4ff-1de9074b61bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:51.148773 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f820a-f7ad-4fbb-a4ff-1de9074b61bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:51.149413 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f820a-f7ad-4fbb-a4ff-1de9074b61bd admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:51.149413 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f820a-f7ad-4fbb-a4ff-1de9074b61bd admin admin] PATH_INFO: `/v3/users/ed8d715515a9427e8737413999aaf38f` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:51.151490 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260f820a-f7ad-4fbb-a4ff-1de9074b61bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed8d715515a9427e8737413999aaf38f)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:51.152572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260f820a-f7ad-4fbb-a4ff-1de9074b61bd admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:51.236891 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-260f820a-f7ad-4fbb-a4ff-1de9074b61bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed8d715515a9427e8737413999aaf38f'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:51.274184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-260f820a-f7ad-4fbb-a4ff-1de9074b61bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed8d715515a9427e8737413999aaf38f'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:51.308217 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-260f820a-f7ad-4fbb-a4ff-1de9074b61bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed8d715515a9427e8737413999aaf38f'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:51.309383 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-260f820a-f7ad-4fbb-a4ff-1de9074b61bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed8d715515a9427e8737413999aaf38f'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:51.310028 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-260f820a-f7ad-4fbb-a4ff-1de9074b61bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed8d715515a9427e8737413999aaf38f'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:51.335628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1188/2384] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:57:51 2022] DELETE /identity/v3/users/ed8d715515a9427e8737413999aaf38f => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:57:51.834912 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8659e5bc-73c6-46d6-b08b-e1ec1dca29cf admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:51.835503 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf8a13-f3f4-4fe2-9c96-37afe05cea1b None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:51.835886 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf8a13-f3f4-4fe2-9c96-37afe05cea1b None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:51.836310 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf8a13-f3f4-4fe2-9c96-37afe05cea1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:51.837188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-8659e5bc-73c6-46d6-b08b-e1ec1dca29cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:51.856716 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8659e5bc-73c6-46d6-b08b-e1ec1dca29cf admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:51.915595 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-d7cf8a13-f3f4-4fe2-9c96-37afe05cea1b None None] MFA Rules not processed for user `da22fc66a1d14fa983463331de110258`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:57:51.917000 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-d7cf8a13-f3f4-4fe2-9c96-37afe05cea1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:51.935100 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fee036-620d-4cb6-899d-a0587a675e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:51.937253 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fee036-620d-4cb6-899d-a0587a675e10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:51.937566 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fee036-620d-4cb6-899d-a0587a675e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:51.937847 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fee036-620d-4cb6-899d-a0587a675e10 admin admin] PATH_INFO: `/v3/projects/ada67ff872b54f72a0c5aa8ca570c5bf` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:51.939159 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fee036-620d-4cb6-899d-a0587a675e10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ada67ff872b54f72a0c5aa8ca570c5bf)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:51.941080 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fee036-620d-4cb6-899d-a0587a675e10 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:51.964850 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:57:51.995098 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1189/2385] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:57:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:57:52.025798 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-68fee036-620d-4cb6-899d-a0587a675e10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ada67ff872b54f72a0c5aa8ca570c5bf', 'request_id': 'req-68fee036-620d-4cb6-899d-a0587a675e10'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:52.035057 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-68fee036-620d-4cb6-899d-a0587a675e10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ada67ff872b54f72a0c5aa8ca570c5bf', 'request_id': 'req-68fee036-620d-4cb6-899d-a0587a675e10'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:52.048935 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-68fee036-620d-4cb6-899d-a0587a675e10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ada67ff872b54f72a0c5aa8ca570c5bf', 'request_id': 'req-68fee036-620d-4cb6-899d-a0587a675e10'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:52.065365 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-68fee036-620d-4cb6-899d-a0587a675e10 admin admin] The token cache is being invalidate because project ada67ff872b54f72a0c5aa8ca570c5bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:57:52.065733 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-68fee036-620d-4cb6-899d-a0587a675e10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:52.066468 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-68fee036-620d-4cb6-899d-a0587a675e10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:52.072141 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1197/2386] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:57:51 2022] DELETE /identity/v3/projects/ada67ff872b54f72a0c5aa8ca570c5bf => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:57:52.073538 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c90f17-008f-4926-8c06-202bb7da1e5f None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:52.074099 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c90f17-008f-4926-8c06-202bb7da1e5f None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:52.074404 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c90f17-008f-4926-8c06-202bb7da1e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:52.112451 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-70c90f17-008f-4926-8c06-202bb7da1e5f None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:57:52.114267 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-70c90f17-008f-4926-8c06-202bb7da1e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:52.205412 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1190/2387] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:57:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:57:52.212984 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30db9f3-76b5-41f8-a776-946c2906003a None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:52.213417 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30db9f3-76b5-41f8-a776-946c2906003a None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:52.213697 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30db9f3-76b5-41f8-a776-946c2906003a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:52.245690 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-c30db9f3-76b5-41f8-a776-946c2906003a None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:57:52.248619 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-c30db9f3-76b5-41f8-a776-946c2906003a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:52.264045 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1198/2388] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:57:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:57:52.268658 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c90f17-008f-4926-8c06-202bb7da1e5f None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:52.280245 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c90f17-008f-4926-8c06-202bb7da1e5f None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:52.285810 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fba3c4-1978-4523-8589-3b81ea74bae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:52.286710 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fba3c4-1978-4523-8589-3b81ea74bae6 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:52.287086 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fba3c4-1978-4523-8589-3b81ea74bae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:52.287702 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fba3c4-1978-4523-8589-3b81ea74bae6 admin admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:52.288261 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fba3c4-1978-4523-8589-3b81ea74bae6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:57:52.288672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fba3c4-1978-4523-8589-3b81ea74bae6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:52.290450 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fba3c4-1978-4523-8589-3b81ea74bae6 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:52.307520 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1191/2389] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:57:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:57:52.312877 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30db9f3-76b5-41f8-a776-946c2906003a None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:52.323977 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30db9f3-76b5-41f8-a776-946c2906003a None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:52.329126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158687df-edbe-42fb-8775-995266365e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:52.329784 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158687df-edbe-42fb-8775-995266365e53 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:52.330060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158687df-edbe-42fb-8775-995266365e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:52.330329 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158687df-edbe-42fb-8775-995266365e53 admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:52.331225 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158687df-edbe-42fb-8775-995266365e53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:52.333009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158687df-edbe-42fb-8775-995266365e53 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:52.338105 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:57:52.386491 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1199/2390] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:57:52 2022] POST /identity/v3/projects => generated 393 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:57:52.390941 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fba3c4-1978-4523-8589-3b81ea74bae6 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:52.404629 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fba3c4-1978-4523-8589-3b81ea74bae6 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:52.497161 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79e6a0-b6aa-42cc-b639-b27a999f2807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:52.498544 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79e6a0-b6aa-42cc-b639-b27a999f2807 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:52.499153 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79e6a0-b6aa-42cc-b639-b27a999f2807 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:52.499714 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79e6a0-b6aa-42cc-b639-b27a999f2807 admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:52.500495 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79e6a0-b6aa-42cc-b639-b27a999f2807 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:52.502969 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79e6a0-b6aa-42cc-b639-b27a999f2807 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:52.529329 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1192/2391] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:57:52 2022] POST /identity/v3/users => generated 297 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:57:52.532982 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158687df-edbe-42fb-8775-995266365e53 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:52.545660 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158687df-edbe-42fb-8775-995266365e53 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:52.551005 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e73e56a-74d3-41e1-8a90-212a3073cf8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:52.552178 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73e56a-74d3-41e1-8a90-212a3073cf8b admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:52.552178 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73e56a-74d3-41e1-8a90-212a3073cf8b admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:52.552405 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73e56a-74d3-41e1-8a90-212a3073cf8b admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:52.553094 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e73e56a-74d3-41e1-8a90-212a3073cf8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:52.554840 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e73e56a-74d3-41e1-8a90-212a3073cf8b admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:52.569341 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1200/2392] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:57:52.572771 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79e6a0-b6aa-42cc-b639-b27a999f2807 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:52.585223 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79e6a0-b6aa-42cc-b639-b27a999f2807 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:52.590900 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd352167-7dc2-474d-a2fb-aa3734af681e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:52.591883 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd352167-7dc2-474d-a2fb-aa3734af681e admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:52.592518 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd352167-7dc2-474d-a2fb-aa3734af681e admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:52.592984 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd352167-7dc2-474d-a2fb-aa3734af681e admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:52.593759 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd352167-7dc2-474d-a2fb-aa3734af681e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:52.595480 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd352167-7dc2-474d-a2fb-aa3734af681e admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:52.607805 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1193/2393] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 22:57:52.612043 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e73e56a-74d3-41e1-8a90-212a3073cf8b admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:52.625489 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e73e56a-74d3-41e1-8a90-212a3073cf8b admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:52.630822 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347b225-cc33-40db-b312-eb73e39135e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:52.631538 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347b225-cc33-40db-b312-eb73e39135e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:52.631959 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347b225-cc33-40db-b312-eb73e39135e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:52.632505 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347b225-cc33-40db-b312-eb73e39135e8 admin admin] PATH_INFO: `/v3/projects/c461f878c6a34314a321e74e4aa2a902/users/9dab69bf5fc1410d829f0da4c40e4585/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:52.650998 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347b225-cc33-40db-b312-eb73e39135e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c461f878c6a34314a321e74e4aa2a902, user_id=9dab69bf5fc1410d829f0da4c40e4585, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:52.653812 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347b225-cc33-40db-b312-eb73e39135e8 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:52.675169 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1201/2394] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:57:52 2022] PUT /identity/v3/projects/c461f878c6a34314a321e74e4aa2a902/users/9dab69bf5fc1410d829f0da4c40e4585/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:57:52.687262 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd759644-cf43-4c26-b535-6b9d3fd1da44 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:52.687549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd759644-cf43-4c26-b535-6b9d3fd1da44 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:52.687826 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd759644-cf43-4c26-b535-6b9d3fd1da44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:52.725828 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-fd759644-cf43-4c26-b535-6b9d3fd1da44 None None] MFA Rules not processed for user `9dab69bf5fc1410d829f0da4c40e4585`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:57:52.727121 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-fd759644-cf43-4c26-b535-6b9d3fd1da44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:52.780831 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:57:52.782500 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347b225-cc33-40db-b312-eb73e39135e8 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:52.784538 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-1347b225-cc33-40db-b312-eb73e39135e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:52.816446 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1194/2395] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:57:52 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:57:52.820992 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd759644-cf43-4c26-b535-6b9d3fd1da44 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:52.857180 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347b225-cc33-40db-b312-eb73e39135e8 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:52.862430 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f300140-de67-4c2f-abdd-87b09b6b7a28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:52.863293 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f300140-de67-4c2f-abdd-87b09b6b7a28 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:52.863569 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f300140-de67-4c2f-abdd-87b09b6b7a28 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:52.863841 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f300140-de67-4c2f-abdd-87b09b6b7a28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:52.865745 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-9f300140-de67-4c2f-abdd-87b09b6b7a28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:52.878616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f300140-de67-4c2f-abdd-87b09b6b7a28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:52.880216 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f300140-de67-4c2f-abdd-87b09b6b7a28 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:52.890226 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1202/2396] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:57:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:57:52.902589 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd759644-cf43-4c26-b535-6b9d3fd1da44 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:52.908762 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c0d30-bfd6-421a-9a6b-821b0a7f6a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:52.909411 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c0d30-bfd6-421a-9a6b-821b0a7f6a4d admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:52.909687 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c0d30-bfd6-421a-9a6b-821b0a7f6a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:52.909957 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c0d30-bfd6-421a-9a6b-821b0a7f6a4d admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:52.910577 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76c0d30-bfd6-421a-9a6b-821b0a7f6a4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:52.912498 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76c0d30-bfd6-421a-9a6b-821b0a7f6a4d admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:52.960775 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1195/2397] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:57:52 2022] POST /identity/v3/projects => generated 391 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:57:52.965957 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f300140-de67-4c2f-abdd-87b09b6b7a28 service neutron] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:52.981048 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f300140-de67-4c2f-abdd-87b09b6b7a28 service neutron] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:53.057858 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16550f5f-ec1d-4759-a36c-1ed58f87bce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:53.058948 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16550f5f-ec1d-4759-a36c-1ed58f87bce8 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:53.058948 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16550f5f-ec1d-4759-a36c-1ed58f87bce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:53.059210 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16550f5f-ec1d-4759-a36c-1ed58f87bce8 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:53.059856 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16550f5f-ec1d-4759-a36c-1ed58f87bce8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:53.062403 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16550f5f-ec1d-4759-a36c-1ed58f87bce8 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:53.095822 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1203/2398] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:57:52 2022] POST /identity/v3/users => generated 302 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:57:53.100074 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c0d30-bfd6-421a-9a6b-821b0a7f6a4d admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:53.114300 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c0d30-bfd6-421a-9a6b-821b0a7f6a4d admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:53.121946 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98c00d-192c-477a-9cc8-db15b0f7c6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:53.122262 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98c00d-192c-477a-9cc8-db15b0f7c6e0 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:53.122549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98c00d-192c-477a-9cc8-db15b0f7c6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:53.123446 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98c00d-192c-477a-9cc8-db15b0f7c6e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:53.124875 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a98c00d-192c-477a-9cc8-db15b0f7c6e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:53.126755 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a98c00d-192c-477a-9cc8-db15b0f7c6e0 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:53.143319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1196/2399] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:57:53 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:57:53.148440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16550f5f-ec1d-4759-a36c-1ed58f87bce8 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:53.162367 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16550f5f-ec1d-4759-a36c-1ed58f87bce8 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:53.167761 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b515fb1c-0c9a-4e8d-b685-be928dd3a868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:53.168618 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b515fb1c-0c9a-4e8d-b685-be928dd3a868 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:53.168905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b515fb1c-0c9a-4e8d-b685-be928dd3a868 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:53.169234 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b515fb1c-0c9a-4e8d-b685-be928dd3a868 admin admin] PATH_INFO: `/v3/projects/1b2956cb339849aba432ac0cf5109704/users/814cb3c562a84f14bc29954809e7cd58/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:53.191350 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b515fb1c-0c9a-4e8d-b685-be928dd3a868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b2956cb339849aba432ac0cf5109704, user_id=814cb3c562a84f14bc29954809e7cd58, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:53.194636 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b515fb1c-0c9a-4e8d-b685-be928dd3a868 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:53.212974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1204/2400] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:57:53 2022] PUT /identity/v3/projects/1b2956cb339849aba432ac0cf5109704/users/814cb3c562a84f14bc29954809e7cd58/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:57:53.217814 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98c00d-192c-477a-9cc8-db15b0f7c6e0 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:53.229591 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98c00d-192c-477a-9cc8-db15b0f7c6e0 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:53.347726 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bd8463-7360-4cba-b7bc-c76124ddc397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:53.348840 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd8463-7360-4cba-b7bc-c76124ddc397 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:53.350382 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd8463-7360-4cba-b7bc-c76124ddc397 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:53.350833 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd8463-7360-4cba-b7bc-c76124ddc397 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:53.351644 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bd8463-7360-4cba-b7bc-c76124ddc397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:53.353141 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b515fb1c-0c9a-4e8d-b685-be928dd3a868 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:53.353701 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bd8463-7360-4cba-b7bc-c76124ddc397 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:53.355005 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-b515fb1c-0c9a-4e8d-b685-be928dd3a868 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:53.371478 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1197/2401] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:57:53 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 22:57:53.376044 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bd8463-7360-4cba-b7bc-c76124ddc397 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:53.380761 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:57:53.398448 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bd8463-7360-4cba-b7bc-c76124ddc397 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:53.412857 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff021385-64f8-4a49-9fc4-f6d65f790d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:53.414102 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff021385-64f8-4a49-9fc4-f6d65f790d7d admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:53.414404 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff021385-64f8-4a49-9fc4-f6d65f790d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:53.414686 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff021385-64f8-4a49-9fc4-f6d65f790d7d admin admin] PATH_INFO: `/v3/projects/1b2956cb339849aba432ac0cf5109704/users/814cb3c562a84f14bc29954809e7cd58/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:53.417248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff021385-64f8-4a49-9fc4-f6d65f790d7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b2956cb339849aba432ac0cf5109704, user_id=814cb3c562a84f14bc29954809e7cd58, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:53.420807 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff021385-64f8-4a49-9fc4-f6d65f790d7d admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:53.450154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1198/2402] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:57:53 2022] PUT /identity/v3/projects/1b2956cb339849aba432ac0cf5109704/users/814cb3c562a84f14bc29954809e7cd58/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:57:53.461400 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde3304-df20-4ed4-afc6-23ad40423206 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:53.461400 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde3304-df20-4ed4-afc6-23ad40423206 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:53.461729 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde3304-df20-4ed4-afc6-23ad40423206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:53.497249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-9dde3304-df20-4ed4-afc6-23ad40423206 None None] MFA Rules not processed for user `814cb3c562a84f14bc29954809e7cd58`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:57:53.497485 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-9dde3304-df20-4ed4-afc6-23ad40423206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:53.502361 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b515fb1c-0c9a-4e8d-b685-be928dd3a868 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:53.532847 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:57:53.587914 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8658d96-02d9-4263-869a-8d7de8b78fe4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f46dee0f73e49eab54c11556e3b86bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:53.589347 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8658d96-02d9-4263-869a-8d7de8b78fe4 service placement] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:53.589347 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8658d96-02d9-4263-869a-8d7de8b78fe4 service placement] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:53.589731 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8658d96-02d9-4263-869a-8d7de8b78fe4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:53.590839 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1199/2403] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:57:53 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:57:53.595927 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-d8658d96-02d9-4263-869a-8d7de8b78fe4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:53.602311 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde3304-df20-4ed4-afc6-23ad40423206 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:53.604278 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-9dde3304-df20-4ed4-afc6-23ad40423206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:53.612352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8658d96-02d9-4263-869a-8d7de8b78fe4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:53.614497 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8658d96-02d9-4263-869a-8d7de8b78fe4 service placement] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:53.629433 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1205/2404] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:57:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 22:57:53.677560 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde3304-df20-4ed4-afc6-23ad40423206 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:53.683770 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b968d94-95c9-4195-b796-653aec715837 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:53.684644 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b968d94-95c9-4195-b796-653aec715837 service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:53.685003 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b968d94-95c9-4195-b796-653aec715837 service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:53.685425 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b968d94-95c9-4195-b796-653aec715837 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:53.696867 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b968d94-95c9-4195-b796-653aec715837 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:53.698514 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b968d94-95c9-4195-b796-653aec715837 service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:53.710445 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1200/2405] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:57:53 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 22:57:53.793334 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8658d96-02d9-4263-869a-8d7de8b78fe4 service placement] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:53.796508 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-d8658d96-02d9-4263-869a-8d7de8b78fe4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:53.815192 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8658d96-02d9-4263-869a-8d7de8b78fe4 service placement] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:53.821236 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ab46b-7232-4339-8cbc-b4e6c2cd84dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:53.822121 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ab46b-7232-4339-8cbc-b4e6c2cd84dd service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:53.822401 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ab46b-7232-4339-8cbc-b4e6c2cd84dd service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:53.822802 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ab46b-7232-4339-8cbc-b4e6c2cd84dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:53.927984 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290ab46b-7232-4339-8cbc-b4e6c2cd84dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:53.930045 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290ab46b-7232-4339-8cbc-b4e6c2cd84dd service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:53.945629 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1206/2406] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:57:53 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:57:56.246379 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b968d94-95c9-4195-b796-653aec715837 service cinder] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:56.248936 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-0b968d94-95c9-4195-b796-653aec715837 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:56.338990 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b968d94-95c9-4195-b796-653aec715837 service cinder] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:56.345204 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7664e6-4255-4585-b5e8-a4398b04c8b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:56.346662 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7664e6-4255-4585-b5e8-a4398b04c8b3 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:56.347036 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7664e6-4255-4585-b5e8-a4398b04c8b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:56.347394 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7664e6-4255-4585-b5e8-a4398b04c8b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:56.349367 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-3d7664e6-4255-4585-b5e8-a4398b04c8b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:56.362655 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7664e6-4255-4585-b5e8-a4398b04c8b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:56.365244 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7664e6-4255-4585-b5e8-a4398b04c8b3 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:56.377677 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1201/2407] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:57:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:57:58.081759 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ab46b-7232-4339-8cbc-b4e6c2cd84dd service cinder] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:58.164503 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:57:58.209127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ab46b-7232-4339-8cbc-b4e6c2cd84dd service cinder] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:58.220686 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dcb2da-7d2e-4063-8fed-933aa0188d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:58.221466 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dcb2da-7d2e-4063-8fed-933aa0188d8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:58.221737 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dcb2da-7d2e-4063-8fed-933aa0188d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:58.222012 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dcb2da-7d2e-4063-8fed-933aa0188d8d admin admin] PATH_INFO: `/v3/users/9dab69bf5fc1410d829f0da4c40e4585` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:58.223165 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dcb2da-7d2e-4063-8fed-933aa0188d8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dab69bf5fc1410d829f0da4c40e4585)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:58.226000 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dcb2da-7d2e-4063-8fed-933aa0188d8d admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:58.298215 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-f2dcb2da-7d2e-4063-8fed-933aa0188d8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dab69bf5fc1410d829f0da4c40e4585'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:58.313103 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-f2dcb2da-7d2e-4063-8fed-933aa0188d8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dab69bf5fc1410d829f0da4c40e4585'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:58.322710 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-f2dcb2da-7d2e-4063-8fed-933aa0188d8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dab69bf5fc1410d829f0da4c40e4585'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:58.323410 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-f2dcb2da-7d2e-4063-8fed-933aa0188d8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dab69bf5fc1410d829f0da4c40e4585'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:58.324011 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-f2dcb2da-7d2e-4063-8fed-933aa0188d8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dab69bf5fc1410d829f0da4c40e4585'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:58.345190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1207/2408] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:57:58 2022] DELETE /identity/v3/users/9dab69bf5fc1410d829f0da4c40e4585 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:57:58.349686 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a9ec7-d957-4692-bb25-c56f5e708854 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:58.350163 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a9ec7-d957-4692-bb25-c56f5e708854 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:58.350251 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a9ec7-d957-4692-bb25-c56f5e708854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:58.373120 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-f45a9ec7-d957-4692-bb25-c56f5e708854 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:57:58.374550 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-f45a9ec7-d957-4692-bb25-c56f5e708854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:58.457727 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1202/2409] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:57:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:57:58.468316 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dcb2da-7d2e-4063-8fed-933aa0188d8d admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:58.469846 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-f2dcb2da-7d2e-4063-8fed-933aa0188d8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:58.535102 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dcb2da-7d2e-4063-8fed-933aa0188d8d admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:58.539552 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be4824-5f26-48fd-9efc-776e1ca9eaf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:58.540176 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be4824-5f26-48fd-9efc-776e1ca9eaf2 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:58.540456 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be4824-5f26-48fd-9efc-776e1ca9eaf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:58.540731 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be4824-5f26-48fd-9efc-776e1ca9eaf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:58.552840 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05be4824-5f26-48fd-9efc-776e1ca9eaf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:58.554536 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05be4824-5f26-48fd-9efc-776e1ca9eaf2 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:58.565518 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1208/2410] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:57:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:57:58.909159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a9ec7-d957-4692-bb25-c56f5e708854 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:58.911186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-f45a9ec7-d957-4692-bb25-c56f5e708854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:58.958275 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a9ec7-d957-4692-bb25-c56f5e708854 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:58.975705 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07733d58-a8b8-4c17-9072-9597ec4c703a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:58.976638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07733d58-a8b8-4c17-9072-9597ec4c703a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:58.977017 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07733d58-a8b8-4c17-9072-9597ec4c703a admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:58.977421 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07733d58-a8b8-4c17-9072-9597ec4c703a admin admin] PATH_INFO: `/v3/projects/c461f878c6a34314a321e74e4aa2a902` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:58.980763 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07733d58-a8b8-4c17-9072-9597ec4c703a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c461f878c6a34314a321e74e4aa2a902)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:58.982889 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07733d58-a8b8-4c17-9072-9597ec4c703a admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:58.988744 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:57:59.070261 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-07733d58-a8b8-4c17-9072-9597ec4c703a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c461f878c6a34314a321e74e4aa2a902', 'request_id': 'req-07733d58-a8b8-4c17-9072-9597ec4c703a'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:59.088287 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-07733d58-a8b8-4c17-9072-9597ec4c703a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c461f878c6a34314a321e74e4aa2a902', 'request_id': 'req-07733d58-a8b8-4c17-9072-9597ec4c703a'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:59.097496 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-07733d58-a8b8-4c17-9072-9597ec4c703a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c461f878c6a34314a321e74e4aa2a902', 'request_id': 'req-07733d58-a8b8-4c17-9072-9597ec4c703a'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:59.130673 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-07733d58-a8b8-4c17-9072-9597ec4c703a admin admin] The token cache is being invalidate because project c461f878c6a34314a321e74e4aa2a902 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:57:59.131066 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-07733d58-a8b8-4c17-9072-9597ec4c703a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:59.132048 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-07733d58-a8b8-4c17-9072-9597ec4c703a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:59.136520 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1203/2411] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:57:58 2022] DELETE /identity/v3/projects/c461f878c6a34314a321e74e4aa2a902 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:57:59.138046 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be4824-5f26-48fd-9efc-776e1ca9eaf2 service neutron] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:59.151536 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be4824-5f26-48fd-9efc-776e1ca9eaf2 service neutron] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:59.224079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99559ca2-5813-494c-9a02-332c0cf3ca18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:59.224791 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99559ca2-5813-494c-9a02-332c0cf3ca18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:59.225139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99559ca2-5813-494c-9a02-332c0cf3ca18 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:59.225522 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99559ca2-5813-494c-9a02-332c0cf3ca18 admin admin] PATH_INFO: `/v3/users/814cb3c562a84f14bc29954809e7cd58` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:59.228892 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99559ca2-5813-494c-9a02-332c0cf3ca18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=814cb3c562a84f14bc29954809e7cd58)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:59.232594 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99559ca2-5813-494c-9a02-332c0cf3ca18 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:59.241129 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:57:59.304756 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-99559ca2-5813-494c-9a02-332c0cf3ca18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '814cb3c562a84f14bc29954809e7cd58'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:59.322368 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-99559ca2-5813-494c-9a02-332c0cf3ca18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '814cb3c562a84f14bc29954809e7cd58'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:59.345401 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-99559ca2-5813-494c-9a02-332c0cf3ca18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '814cb3c562a84f14bc29954809e7cd58'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:59.347121 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-99559ca2-5813-494c-9a02-332c0cf3ca18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '814cb3c562a84f14bc29954809e7cd58'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:59.349000 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-99559ca2-5813-494c-9a02-332c0cf3ca18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '814cb3c562a84f14bc29954809e7cd58'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:59.386162 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1209/2412] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:57:59 2022] DELETE /identity/v3/users/814cb3c562a84f14bc29954809e7cd58 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:57:59.703334 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07733d58-a8b8-4c17-9072-9597ec4c703a admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:57:59.705439 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-07733d58-a8b8-4c17-9072-9597ec4c703a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:59.722652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07733d58-a8b8-4c17-9072-9597ec4c703a admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:57:59.787318 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc07be5-9ffa-47f6-92b5-e1bbf5669a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:57:59.787951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc07be5-9ffa-47f6-92b5-e1bbf5669a6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:59.788228 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc07be5-9ffa-47f6-92b5-e1bbf5669a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:59.788500 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc07be5-9ffa-47f6-92b5-e1bbf5669a6f admin admin] PATH_INFO: `/v3/projects/1b2956cb339849aba432ac0cf5109704` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:59.789620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc07be5-9ffa-47f6-92b5-e1bbf5669a6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b2956cb339849aba432ac0cf5109704)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:57:59.791513 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc07be5-9ffa-47f6-92b5-e1bbf5669a6f admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:57:59.837736 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-4cc07be5-9ffa-47f6-92b5-e1bbf5669a6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b2956cb339849aba432ac0cf5109704', 'request_id': 'req-4cc07be5-9ffa-47f6-92b5-e1bbf5669a6f'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:59.845429 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-4cc07be5-9ffa-47f6-92b5-e1bbf5669a6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b2956cb339849aba432ac0cf5109704', 'request_id': 'req-4cc07be5-9ffa-47f6-92b5-e1bbf5669a6f'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:59.850991 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-4cc07be5-9ffa-47f6-92b5-e1bbf5669a6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b2956cb339849aba432ac0cf5109704', 'request_id': 'req-4cc07be5-9ffa-47f6-92b5-e1bbf5669a6f'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:59.863752 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-4cc07be5-9ffa-47f6-92b5-e1bbf5669a6f admin admin] The token cache is being invalidate because project 1b2956cb339849aba432ac0cf5109704 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:57:59.864037 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-4cc07be5-9ffa-47f6-92b5-e1bbf5669a6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:59.864672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-4cc07be5-9ffa-47f6-92b5-e1bbf5669a6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:57:59.866178 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1204/2413] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:57:59 2022] DELETE /identity/v3/projects/1b2956cb339849aba432ac0cf5109704 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:57:59.870965 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f254ce-ec5f-429f-a736-477ffec2d222 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:59.871255 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f254ce-ec5f-429f-a736-477ffec2d222 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:59.871539 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f254ce-ec5f-429f-a736-477ffec2d222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:59.892168 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-51f254ce-ec5f-429f-a736-477ffec2d222 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:57:59.893391 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-51f254ce-ec5f-429f-a736-477ffec2d222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:59.950706 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1210/2414] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:57:59.959303 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b17db-185b-4572-8032-9d77c8d0cbad None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:57:59.959590 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b17db-185b-4572-8032-9d77c8d0cbad None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:57:59.959856 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b17db-185b-4572-8032-9d77c8d0cbad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:57:59.979391 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-cb2b17db-185b-4572-8032-9d77c8d0cbad None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:57:59.980593 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-cb2b17db-185b-4572-8032-9d77c8d0cbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:57:59.989021 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1205/2415] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:57:59.993390 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f254ce-ec5f-429f-a736-477ffec2d222 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:00.003498 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f254ce-ec5f-429f-a736-477ffec2d222 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:00.008305 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d47dd9-ff42-4180-a512-186d3f9960fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:00.008901 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d47dd9-ff42-4180-a512-186d3f9960fe admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:00.009179 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d47dd9-ff42-4180-a512-186d3f9960fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:00.009481 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d47dd9-ff42-4180-a512-186d3f9960fe admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:00.010119 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d47dd9-ff42-4180-a512-186d3f9960fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:58:00.010418 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d47dd9-ff42-4180-a512-186d3f9960fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:00.012066 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d47dd9-ff42-4180-a512-186d3f9960fe admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:00.023383 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1211/2416] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:58:00.027562 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b17db-185b-4572-8032-9d77c8d0cbad None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:00.037260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b17db-185b-4572-8032-9d77c8d0cbad None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:00.041834 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6228e-97de-41a9-9f1e-4fe722425376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:00.042444 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6228e-97de-41a9-9f1e-4fe722425376 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:00.042717 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6228e-97de-41a9-9f1e-4fe722425376 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:00.042982 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6228e-97de-41a9-9f1e-4fe722425376 admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:00.043657 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc6228e-97de-41a9-9f1e-4fe722425376 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:00.047072 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc6228e-97de-41a9-9f1e-4fe722425376 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:00.061510 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:58:00.075402 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1206/2417] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:58:00 2022] POST /identity/v3/projects => generated 399 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:58:00.079455 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d47dd9-ff42-4180-a512-186d3f9960fe admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:00.089556 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d47dd9-ff42-4180-a512-186d3f9960fe admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:00.164768 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f738f51-0c29-4c38-bac1-037faf717559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:00.165590 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f738f51-0c29-4c38-bac1-037faf717559 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:00.165983 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f738f51-0c29-4c38-bac1-037faf717559 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:00.166367 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f738f51-0c29-4c38-bac1-037faf717559 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:00.167166 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f738f51-0c29-4c38-bac1-037faf717559 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:00.169906 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f738f51-0c29-4c38-bac1-037faf717559 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:00.186436 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:58:00.195287 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1212/2418] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:58:00 2022] POST /identity/v3/users => generated 300 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:58:00.202016 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6228e-97de-41a9-9f1e-4fe722425376 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:00.212839 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6228e-97de-41a9-9f1e-4fe722425376 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:00.218411 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facdb6ae-92d2-486b-a29c-b504c8c5d734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:00.219055 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdb6ae-92d2-486b-a29c-b504c8c5d734 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:00.219356 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdb6ae-92d2-486b-a29c-b504c8c5d734 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:00.219624 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdb6ae-92d2-486b-a29c-b504c8c5d734 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:00.220262 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facdb6ae-92d2-486b-a29c-b504c8c5d734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:00.221946 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facdb6ae-92d2-486b-a29c-b504c8c5d734 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:00.232770 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1207/2419] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:58:00.236755 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f738f51-0c29-4c38-bac1-037faf717559 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:00.247718 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f738f51-0c29-4c38-bac1-037faf717559 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:00.253142 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca22846-3f7b-404a-b47a-c0852feacab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:00.253865 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca22846-3f7b-404a-b47a-c0852feacab6 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:00.254642 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca22846-3f7b-404a-b47a-c0852feacab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:00.254642 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca22846-3f7b-404a-b47a-c0852feacab6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:00.255098 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca22846-3f7b-404a-b47a-c0852feacab6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:00.256979 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca22846-3f7b-404a-b47a-c0852feacab6 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:00.270221 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1213/2420] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:58:00.274436 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facdb6ae-92d2-486b-a29c-b504c8c5d734 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:00.285128 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facdb6ae-92d2-486b-a29c-b504c8c5d734 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:00.289813 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8412f95f-eafe-4bd2-8bf8-c956f5167f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:00.290657 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8412f95f-eafe-4bd2-8bf8-c956f5167f11 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:00.290937 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8412f95f-eafe-4bd2-8bf8-c956f5167f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:00.291206 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8412f95f-eafe-4bd2-8bf8-c956f5167f11 admin admin] PATH_INFO: `/v3/projects/cfacc1bb5e934c9aa5ebba5a75d15ab2/users/a0fe58c54faf4f32a7c3a5db48ff396a/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:00.309271 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8412f95f-eafe-4bd2-8bf8-c956f5167f11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfacc1bb5e934c9aa5ebba5a75d15ab2, user_id=a0fe58c54faf4f32a7c3a5db48ff396a, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:00.312405 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8412f95f-eafe-4bd2-8bf8-c956f5167f11 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:00.324893 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1208/2421] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:58:00 2022] PUT /identity/v3/projects/cfacc1bb5e934c9aa5ebba5a75d15ab2/users/a0fe58c54faf4f32a7c3a5db48ff396a/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:58:00.390300 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536b491-8bb2-489c-be76-c51e70eeb1d7 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:00.390969 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536b491-8bb2-489c-be76-c51e70eeb1d7 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:00.391250 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536b491-8bb2-489c-be76-c51e70eeb1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:00.421136 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-9536b491-8bb2-489c-be76-c51e70eeb1d7 None None] MFA Rules not processed for user `a0fe58c54faf4f32a7c3a5db48ff396a`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:58:00.422495 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-9536b491-8bb2-489c-be76-c51e70eeb1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:00.494503 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1214/2422] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:58:00 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:58:00.499469 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8412f95f-eafe-4bd2-8bf8-c956f5167f11 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:00.567089 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8412f95f-eafe-4bd2-8bf8-c956f5167f11 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:00.572008 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab583bf-5edd-4a75-b4ec-a3f6f93b52d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:00.572632 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab583bf-5edd-4a75-b4ec-a3f6f93b52d3 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:00.572918 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab583bf-5edd-4a75-b4ec-a3f6f93b52d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:00.573197 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab583bf-5edd-4a75-b4ec-a3f6f93b52d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:00.573868 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab583bf-5edd-4a75-b4ec-a3f6f93b52d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:00.577020 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab583bf-5edd-4a75-b4ec-a3f6f93b52d3 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:00.609562 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1209/2423] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:58:00 2022] POST /identity/v3/projects => generated 399 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:58:00.613755 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9536b491-8bb2-489c-be76-c51e70eeb1d7 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:00.630172 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9536b491-8bb2-489c-be76-c51e70eeb1d7 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:00.689747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b7773-a470-4437-b5ef-f3bcd35456a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:00.690435 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b7773-a470-4437-b5ef-f3bcd35456a2 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:00.690715 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b7773-a470-4437-b5ef-f3bcd35456a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:00.690990 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b7773-a470-4437-b5ef-f3bcd35456a2 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:00.691619 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75b7773-a470-4437-b5ef-f3bcd35456a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:00.693580 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75b7773-a470-4437-b5ef-f3bcd35456a2 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:00.715557 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1215/2424] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:58:00 2022] POST /identity/v3/users => generated 306 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:58:00.720778 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab583bf-5edd-4a75-b4ec-a3f6f93b52d3 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:00.731971 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab583bf-5edd-4a75-b4ec-a3f6f93b52d3 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:00.736819 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37505296-395c-4741-ae23-094a81c9014b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:00.737457 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37505296-395c-4741-ae23-094a81c9014b admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:00.737856 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37505296-395c-4741-ae23-094a81c9014b admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:00.738029 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37505296-395c-4741-ae23-094a81c9014b admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:00.738690 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37505296-395c-4741-ae23-094a81c9014b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:00.740360 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37505296-395c-4741-ae23-094a81c9014b admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:00.751745 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1210/2425] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:58:00.756036 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b7773-a470-4437-b5ef-f3bcd35456a2 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:00.768321 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b7773-a470-4437-b5ef-f3bcd35456a2 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:00.773348 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc6dce1-2f70-4f9a-8e29-89aff9dbeef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:00.774025 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6dce1-2f70-4f9a-8e29-89aff9dbeef9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:00.774309 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6dce1-2f70-4f9a-8e29-89aff9dbeef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:00.774587 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6dce1-2f70-4f9a-8e29-89aff9dbeef9 admin admin] PATH_INFO: `/v3/projects/6bcc63082b954355a2aac1f097e60aab/users/2121ca7a7bcb405786ed2a22b85cec21/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:00.791127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc6dce1-2f70-4f9a-8e29-89aff9dbeef9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bcc63082b954355a2aac1f097e60aab, user_id=2121ca7a7bcb405786ed2a22b85cec21, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:00.793982 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc6dce1-2f70-4f9a-8e29-89aff9dbeef9 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:00.807364 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1216/2426] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:58:00 2022] PUT /identity/v3/projects/6bcc63082b954355a2aac1f097e60aab/users/2121ca7a7bcb405786ed2a22b85cec21/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:58:00.812390 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37505296-395c-4741-ae23-094a81c9014b admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:00.822543 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37505296-395c-4741-ae23-094a81c9014b admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:00.883165 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e723e26-4d17-48ea-a919-f826fde8b6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:00.883786 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e723e26-4d17-48ea-a919-f826fde8b6c1 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:00.884065 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e723e26-4d17-48ea-a919-f826fde8b6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:00.884334 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e723e26-4d17-48ea-a919-f826fde8b6c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:00.885034 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e723e26-4d17-48ea-a919-f826fde8b6c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:00.886726 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e723e26-4d17-48ea-a919-f826fde8b6c1 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:00.898476 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1211/2427] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:58:00.902605 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc6dce1-2f70-4f9a-8e29-89aff9dbeef9 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:00.913747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc6dce1-2f70-4f9a-8e29-89aff9dbeef9 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:00.918711 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee41c5d8-1a8d-449a-8c14-2c4af8433de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:00.919488 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41c5d8-1a8d-449a-8c14-2c4af8433de5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:00.919766 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41c5d8-1a8d-449a-8c14-2c4af8433de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:00.920038 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41c5d8-1a8d-449a-8c14-2c4af8433de5 admin admin] PATH_INFO: `/v3/projects/6bcc63082b954355a2aac1f097e60aab/users/2121ca7a7bcb405786ed2a22b85cec21/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:00.921972 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee41c5d8-1a8d-449a-8c14-2c4af8433de5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bcc63082b954355a2aac1f097e60aab, user_id=2121ca7a7bcb405786ed2a22b85cec21, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:00.924667 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee41c5d8-1a8d-449a-8c14-2c4af8433de5 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:00.935896 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1217/2428] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:58:00 2022] PUT /identity/v3/projects/6bcc63082b954355a2aac1f097e60aab/users/2121ca7a7bcb405786ed2a22b85cec21/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:58:00.946382 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2939dfef-d3e1-40cd-a87e-d9c585865e75 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:00.946673 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2939dfef-d3e1-40cd-a87e-d9c585865e75 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:00.946989 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2939dfef-d3e1-40cd-a87e-d9c585865e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:00.979776 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-2939dfef-d3e1-40cd-a87e-d9c585865e75 None None] MFA Rules not processed for user `2121ca7a7bcb405786ed2a22b85cec21`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:58:00.981020 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-2939dfef-d3e1-40cd-a87e-d9c585865e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:01.000980 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:58:01.057702 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1212/2429] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:58:00 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:58:01.068916 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee41c5d8-1a8d-449a-8c14-2c4af8433de5 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:01.070653 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-ee41c5d8-1a8d-449a-8c14-2c4af8433de5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:01.103639 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:58:01.135251 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee41c5d8-1a8d-449a-8c14-2c4af8433de5 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:01.139534 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456eea16-ca55-49de-ac73-ff0c5afb2d1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:01.140292 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456eea16-ca55-49de-ac73-ff0c5afb2d1a service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:01.140624 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456eea16-ca55-49de-ac73-ff0c5afb2d1a service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:01.140846 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456eea16-ca55-49de-ac73-ff0c5afb2d1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:01.150553 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456eea16-ca55-49de-ac73-ff0c5afb2d1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:01.152310 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456eea16-ca55-49de-ac73-ff0c5afb2d1a service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:01.163258 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1218/2430] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:58:01 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:58:01.248852 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2939dfef-d3e1-40cd-a87e-d9c585865e75 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:01.259393 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2939dfef-d3e1-40cd-a87e-d9c585865e75 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:01.264024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06930f72-2edc-490d-8fbd-8bf3800e8cf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:01.264638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06930f72-2edc-490d-8fbd-8bf3800e8cf8 service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:01.264922 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06930f72-2edc-490d-8fbd-8bf3800e8cf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:01.265191 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06930f72-2edc-490d-8fbd-8bf3800e8cf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:01.326791 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06930f72-2edc-490d-8fbd-8bf3800e8cf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:01.328233 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06930f72-2edc-490d-8fbd-8bf3800e8cf8 service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:01.339685 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1213/2431] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:58:01 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:58:05.858140 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456eea16-ca55-49de-ac73-ff0c5afb2d1a service cinder] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:05.930365 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456eea16-ca55-49de-ac73-ff0c5afb2d1a service cinder] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:05.936718 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab018584-859f-4c33-baf1-d6fc76f60664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:05.937380 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab018584-859f-4c33-baf1-d6fc76f60664 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:05.937665 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab018584-859f-4c33-baf1-d6fc76f60664 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:05.937939 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab018584-859f-4c33-baf1-d6fc76f60664 admin admin] PATH_INFO: `/v3/users/a0fe58c54faf4f32a7c3a5db48ff396a` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:05.939448 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab018584-859f-4c33-baf1-d6fc76f60664 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0fe58c54faf4f32a7c3a5db48ff396a)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:05.941772 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab018584-859f-4c33-baf1-d6fc76f60664 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:06.001824 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-ab018584-859f-4c33-baf1-d6fc76f60664 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0fe58c54faf4f32a7c3a5db48ff396a'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:06.011359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-ab018584-859f-4c33-baf1-d6fc76f60664 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0fe58c54faf4f32a7c3a5db48ff396a'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:06.014539 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06930f72-2edc-490d-8fbd-8bf3800e8cf8 service cinder] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:06.016363 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-06930f72-2edc-490d-8fbd-8bf3800e8cf8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:06.020881 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-ab018584-859f-4c33-baf1-d6fc76f60664 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0fe58c54faf4f32a7c3a5db48ff396a'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:06.022222 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-ab018584-859f-4c33-baf1-d6fc76f60664 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0fe58c54faf4f32a7c3a5db48ff396a'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:06.022844 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-ab018584-859f-4c33-baf1-d6fc76f60664 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0fe58c54faf4f32a7c3a5db48ff396a'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:06.048098 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1219/2432] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:58:05 2022] DELETE /identity/v3/users/a0fe58c54faf4f32a7c3a5db48ff396a => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:58:06.050270 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50efcc-fbc0-4a2b-8a6a-1a14bfec8489 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:06.050557 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50efcc-fbc0-4a2b-8a6a-1a14bfec8489 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:06.050834 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50efcc-fbc0-4a2b-8a6a-1a14bfec8489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:06.075713 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-4b50efcc-fbc0-4a2b-8a6a-1a14bfec8489 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:58:06.076727 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-4b50efcc-fbc0-4a2b-8a6a-1a14bfec8489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:06.083502 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06930f72-2edc-490d-8fbd-8bf3800e8cf8 service cinder] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:06.131710 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:58:06.148516 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c521e0b9-8c33-4d52-8f6a-99462fe85018 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc7524edf5d5422f91d92fd91be5ee1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:06.149493 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521e0b9-8c33-4d52-8f6a-99462fe85018 service glance] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:06.149773 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521e0b9-8c33-4d52-8f6a-99462fe85018 service glance] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:06.150377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521e0b9-8c33-4d52-8f6a-99462fe85018 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:06.154178 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-c521e0b9-8c33-4d52-8f6a-99462fe85018 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:06.160541 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1220/2433] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:58:06.173044 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b50efcc-fbc0-4a2b-8a6a-1a14bfec8489 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:06.174507 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-4b50efcc-fbc0-4a2b-8a6a-1a14bfec8489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:06.185632 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:58:06.249854 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b50efcc-fbc0-4a2b-8a6a-1a14bfec8489 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:06.251068 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c521e0b9-8c33-4d52-8f6a-99462fe85018 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:06.252593 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c521e0b9-8c33-4d52-8f6a-99462fe85018 service glance] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:06.253759 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d79764-92a2-4edc-952e-67203e5819d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:06.254377 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d79764-92a2-4edc-952e-67203e5819d4 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:06.254657 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d79764-92a2-4edc-952e-67203e5819d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:06.254929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d79764-92a2-4edc-952e-67203e5819d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:06.265345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d79764-92a2-4edc-952e-67203e5819d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:06.266869 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d79764-92a2-4edc-952e-67203e5819d4 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:06.267548 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1214/2434] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:58:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:58:06.276642 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1221/2435] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:58:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 22:58:06.333465 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c521e0b9-8c33-4d52-8f6a-99462fe85018 service glance] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:06.335766 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-c521e0b9-8c33-4d52-8f6a-99462fe85018 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:06.350180 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c521e0b9-8c33-4d52-8f6a-99462fe85018 service glance] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:06.357095 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f633b22-1586-4a5a-a3f1-6b5e7cb5b458 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:06.358243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f633b22-1586-4a5a-a3f1-6b5e7cb5b458 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:06.358739 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f633b22-1586-4a5a-a3f1-6b5e7cb5b458 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:06.358992 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f633b22-1586-4a5a-a3f1-6b5e7cb5b458 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:06.362808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-3f633b22-1586-4a5a-a3f1-6b5e7cb5b458 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:06.375349 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f633b22-1586-4a5a-a3f1-6b5e7cb5b458 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:06.377563 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f633b22-1586-4a5a-a3f1-6b5e7cb5b458 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:06.391211 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1215/2436] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:58:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:58:06.495844 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d79764-92a2-4edc-952e-67203e5819d4 service neutron] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:06.513096 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d79764-92a2-4edc-952e-67203e5819d4 service neutron] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:06.517210 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f741790-3ca4-4b4c-973f-b84323e11c4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:06.517897 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f741790-3ca4-4b4c-973f-b84323e11c4d service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:06.518137 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f741790-3ca4-4b4c-973f-b84323e11c4d service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:06.518409 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f741790-3ca4-4b4c-973f-b84323e11c4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:06.520310 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-0f741790-3ca4-4b4c-973f-b84323e11c4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:06.540950 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f741790-3ca4-4b4c-973f-b84323e11c4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:06.543423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f741790-3ca4-4b4c-973f-b84323e11c4d service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:06.554932 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1222/2437] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:58:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:58:06.638775 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f633b22-1586-4a5a-a3f1-6b5e7cb5b458 service neutron] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:06.640307 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-3f633b22-1586-4a5a-a3f1-6b5e7cb5b458 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:06.657972 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f633b22-1586-4a5a-a3f1-6b5e7cb5b458 service neutron] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:06.668121 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ece5d-3baa-445b-bcc0-4f30e9a11007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:06.668121 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ece5d-3baa-445b-bcc0-4f30e9a11007 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:06.668121 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ece5d-3baa-445b-bcc0-4f30e9a11007 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:06.668890 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ece5d-3baa-445b-bcc0-4f30e9a11007 admin admin] PATH_INFO: `/v3/projects/cfacc1bb5e934c9aa5ebba5a75d15ab2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:06.674152 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5ece5d-3baa-445b-bcc0-4f30e9a11007 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfacc1bb5e934c9aa5ebba5a75d15ab2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:06.678862 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5ece5d-3baa-445b-bcc0-4f30e9a11007 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:06.743741 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-ae5ece5d-3baa-445b-bcc0-4f30e9a11007 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfacc1bb5e934c9aa5ebba5a75d15ab2', 'request_id': 'req-ae5ece5d-3baa-445b-bcc0-4f30e9a11007'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:06.754024 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-ae5ece5d-3baa-445b-bcc0-4f30e9a11007 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfacc1bb5e934c9aa5ebba5a75d15ab2', 'request_id': 'req-ae5ece5d-3baa-445b-bcc0-4f30e9a11007'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:06.764027 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-ae5ece5d-3baa-445b-bcc0-4f30e9a11007 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfacc1bb5e934c9aa5ebba5a75d15ab2', 'request_id': 'req-ae5ece5d-3baa-445b-bcc0-4f30e9a11007'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:06.776289 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-ae5ece5d-3baa-445b-bcc0-4f30e9a11007 admin admin] The token cache is being invalidate because project cfacc1bb5e934c9aa5ebba5a75d15ab2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:58:06.776681 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-ae5ece5d-3baa-445b-bcc0-4f30e9a11007 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:06.777457 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-ae5ece5d-3baa-445b-bcc0-4f30e9a11007 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:06.779964 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1216/2438] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:58:06 2022] DELETE /identity/v3/projects/cfacc1bb5e934c9aa5ebba5a75d15ab2 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:58:06.783843 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f741790-3ca4-4b4c-973f-b84323e11c4d service neutron] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:06.797538 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f741790-3ca4-4b4c-973f-b84323e11c4d service neutron] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:06.889993 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3711c90-067a-409c-b739-8e02f6293dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:06.891173 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3711c90-067a-409c-b739-8e02f6293dec admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:06.891498 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3711c90-067a-409c-b739-8e02f6293dec admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:06.891884 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3711c90-067a-409c-b739-8e02f6293dec admin admin] PATH_INFO: `/v3/users/2121ca7a7bcb405786ed2a22b85cec21` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:06.893185 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3711c90-067a-409c-b739-8e02f6293dec admin admin] RBAC: Authorizing `identity:delete_user(user_id=2121ca7a7bcb405786ed2a22b85cec21)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:06.895283 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3711c90-067a-409c-b739-8e02f6293dec admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:06.967529 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-d3711c90-067a-409c-b739-8e02f6293dec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2121ca7a7bcb405786ed2a22b85cec21'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:06.986166 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-d3711c90-067a-409c-b739-8e02f6293dec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2121ca7a7bcb405786ed2a22b85cec21'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:06.999870 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-d3711c90-067a-409c-b739-8e02f6293dec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2121ca7a7bcb405786ed2a22b85cec21'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:07.000630 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-d3711c90-067a-409c-b739-8e02f6293dec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2121ca7a7bcb405786ed2a22b85cec21'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:07.001260 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-d3711c90-067a-409c-b739-8e02f6293dec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2121ca7a7bcb405786ed2a22b85cec21'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:07.035311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:58:07.059707 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1223/2439] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:58:06 2022] DELETE /identity/v3/users/2121ca7a7bcb405786ed2a22b85cec21 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:58:07.528309 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ece5d-3baa-445b-bcc0-4f30e9a11007 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:07.530216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-ae5ece5d-3baa-445b-bcc0-4f30e9a11007 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:07.548665 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ece5d-3baa-445b-bcc0-4f30e9a11007 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:07.632655 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4449cf35-f350-44aa-bba0-9d42d364a6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:07.633348 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449cf35-f350-44aa-bba0-9d42d364a6ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:07.633633 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449cf35-f350-44aa-bba0-9d42d364a6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:07.633935 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449cf35-f350-44aa-bba0-9d42d364a6ec admin admin] PATH_INFO: `/v3/projects/6bcc63082b954355a2aac1f097e60aab` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:07.635114 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4449cf35-f350-44aa-bba0-9d42d364a6ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bcc63082b954355a2aac1f097e60aab)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:07.637132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4449cf35-f350-44aa-bba0-9d42d364a6ec admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:07.664557 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:58:07.702651 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-4449cf35-f350-44aa-bba0-9d42d364a6ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bcc63082b954355a2aac1f097e60aab', 'request_id': 'req-4449cf35-f350-44aa-bba0-9d42d364a6ec'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:07.712109 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-4449cf35-f350-44aa-bba0-9d42d364a6ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bcc63082b954355a2aac1f097e60aab', 'request_id': 'req-4449cf35-f350-44aa-bba0-9d42d364a6ec'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:07.721492 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-4449cf35-f350-44aa-bba0-9d42d364a6ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bcc63082b954355a2aac1f097e60aab', 'request_id': 'req-4449cf35-f350-44aa-bba0-9d42d364a6ec'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:07.737604 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-4449cf35-f350-44aa-bba0-9d42d364a6ec admin admin] The token cache is being invalidate because project 6bcc63082b954355a2aac1f097e60aab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:58:07.738262 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-4449cf35-f350-44aa-bba0-9d42d364a6ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:07.739134 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-4449cf35-f350-44aa-bba0-9d42d364a6ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:07.741613 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1217/2440] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:58:07 2022] DELETE /identity/v3/projects/6bcc63082b954355a2aac1f097e60aab => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:58:07.748063 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8722a-1284-4851-abe8-ffc2c21bdafe None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:07.748342 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8722a-1284-4851-abe8-ffc2c21bdafe None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:07.748760 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8722a-1284-4851-abe8-ffc2c21bdafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:07.769704 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-a0a8722a-1284-4851-abe8-ffc2c21bdafe None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:58:07.770923 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-a0a8722a-1284-4851-abe8-ffc2c21bdafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:07.834697 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1224/2441] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:58:07.843976 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa932840-2f3c-43b3-8ee5-cfbbc3272efd None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:07.844260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa932840-2f3c-43b3-8ee5-cfbbc3272efd None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:07.844536 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa932840-2f3c-43b3-8ee5-cfbbc3272efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:07.865131 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-fa932840-2f3c-43b3-8ee5-cfbbc3272efd None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:58:07.866306 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-fa932840-2f3c-43b3-8ee5-cfbbc3272efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:07.875397 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1218/2442] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:58:07.879989 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a8722a-1284-4851-abe8-ffc2c21bdafe None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:07.890213 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a8722a-1284-4851-abe8-ffc2c21bdafe None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:07.895198 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a7dca-aefd-462f-8dcc-11887d8b55be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:07.895859 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a7dca-aefd-462f-8dcc-11887d8b55be admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:07.896141 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a7dca-aefd-462f-8dcc-11887d8b55be admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:07.896412 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a7dca-aefd-462f-8dcc-11887d8b55be admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:07.897059 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a7dca-aefd-462f-8dcc-11887d8b55be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:58:07.897390 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a7dca-aefd-462f-8dcc-11887d8b55be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:07.899000 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a7dca-aefd-462f-8dcc-11887d8b55be admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:07.911032 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1225/2443] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:58:07.915721 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa932840-2f3c-43b3-8ee5-cfbbc3272efd None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:07.926144 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa932840-2f3c-43b3-8ee5-cfbbc3272efd None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:07.930843 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c85cf51-4b0a-41b2-99c7-3a9d3400a7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:07.931442 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85cf51-4b0a-41b2-99c7-3a9d3400a7ba admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:07.931716 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85cf51-4b0a-41b2-99c7-3a9d3400a7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:07.931981 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85cf51-4b0a-41b2-99c7-3a9d3400a7ba admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:07.932622 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c85cf51-4b0a-41b2-99c7-3a9d3400a7ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:07.934635 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c85cf51-4b0a-41b2-99c7-3a9d3400a7ba admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:07.966709 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1219/2444] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:58:07 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:58:07.968116 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a7dca-aefd-462f-8dcc-11887d8b55be admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:07.979088 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a7dca-aefd-462f-8dcc-11887d8b55be admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:08.038711 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14470607-da01-456a-b115-3e8b473ab3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:08.039331 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14470607-da01-456a-b115-3e8b473ab3a5 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:08.039616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14470607-da01-456a-b115-3e8b473ab3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:08.039892 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14470607-da01-456a-b115-3e8b473ab3a5 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:08.040527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14470607-da01-456a-b115-3e8b473ab3a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:08.042491 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14470607-da01-456a-b115-3e8b473ab3a5 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:08.064588 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1226/2445] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:58:07 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:58:08.068911 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c85cf51-4b0a-41b2-99c7-3a9d3400a7ba admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:08.080026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c85cf51-4b0a-41b2-99c7-3a9d3400a7ba admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:08.084634 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39fab9e-3913-4a84-a106-2f7d3424971e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:08.085235 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fab9e-3913-4a84-a106-2f7d3424971e admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:08.085545 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fab9e-3913-4a84-a106-2f7d3424971e admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:08.085815 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fab9e-3913-4a84-a106-2f7d3424971e admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:08.086445 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39fab9e-3913-4a84-a106-2f7d3424971e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:08.088022 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39fab9e-3913-4a84-a106-2f7d3424971e admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:08.098308 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1220/2446] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 22:58:08.102319 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14470607-da01-456a-b115-3e8b473ab3a5 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:08.113203 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14470607-da01-456a-b115-3e8b473ab3a5 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:08.118276 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129e06da-8a61-42e0-981b-caf06b53ca7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:08.118818 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e06da-8a61-42e0-981b-caf06b53ca7a admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:08.119096 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e06da-8a61-42e0-981b-caf06b53ca7a admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:08.119366 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e06da-8a61-42e0-981b-caf06b53ca7a admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:08.120033 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129e06da-8a61-42e0-981b-caf06b53ca7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:08.122378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129e06da-8a61-42e0-981b-caf06b53ca7a admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:08.133736 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1227/2447] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:58:08.137843 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39fab9e-3913-4a84-a106-2f7d3424971e admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:08.148267 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39fab9e-3913-4a84-a106-2f7d3424971e admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:08.154766 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136d8d4-c5ea-4d82-8169-b2f6437cc572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:08.155442 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136d8d4-c5ea-4d82-8169-b2f6437cc572 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:08.156189 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136d8d4-c5ea-4d82-8169-b2f6437cc572 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:08.156189 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136d8d4-c5ea-4d82-8169-b2f6437cc572 admin admin] PATH_INFO: `/v3/projects/1c3a035c71e2483eb86deb3cc6d3eacc/users/11ff134e44b04014afdd13e03ac3ab31/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:08.176305 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f136d8d4-c5ea-4d82-8169-b2f6437cc572 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c3a035c71e2483eb86deb3cc6d3eacc, user_id=11ff134e44b04014afdd13e03ac3ab31, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:08.179158 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f136d8d4-c5ea-4d82-8169-b2f6437cc572 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:08.195814 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1221/2448] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:58:08 2022] PUT /identity/v3/projects/1c3a035c71e2483eb86deb3cc6d3eacc/users/11ff134e44b04014afdd13e03ac3ab31/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:58:08.206653 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c9099-5309-4675-9789-f20bf9a33450 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:08.207711 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c9099-5309-4675-9789-f20bf9a33450 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:08.207711 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c9099-5309-4675-9789-f20bf9a33450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:08.236403 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-a39c9099-5309-4675-9789-f20bf9a33450 None None] MFA Rules not processed for user `11ff134e44b04014afdd13e03ac3ab31`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:58:08.237669 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-a39c9099-5309-4675-9789-f20bf9a33450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:08.310209 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:58:08.321547 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1228/2449] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:58:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:58:08.332629 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136d8d4-c5ea-4d82-8169-b2f6437cc572 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:08.334587 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-f136d8d4-c5ea-4d82-8169-b2f6437cc572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:08.411364 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136d8d4-c5ea-4d82-8169-b2f6437cc572 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:08.418958 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0494d7e5-8f39-4acc-b65c-76decce4d116 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:08.418958 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494d7e5-8f39-4acc-b65c-76decce4d116 service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:08.418958 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494d7e5-8f39-4acc-b65c-76decce4d116 service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:08.418958 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494d7e5-8f39-4acc-b65c-76decce4d116 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:08.438738 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0494d7e5-8f39-4acc-b65c-76decce4d116 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:08.441325 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0494d7e5-8f39-4acc-b65c-76decce4d116 service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:08.452821 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1222/2450] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:58:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:58:08.823055 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28886a3-c086-4cc9-91c4-8493cdb743a9 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:08.824081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28886a3-c086-4cc9-91c4-8493cdb743a9 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:08.824081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28886a3-c086-4cc9-91c4-8493cdb743a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:08.904940 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-b28886a3-c086-4cc9-91c4-8493cdb743a9 None None] MFA Rules not processed for user `aff5e804c64345b298c745287a1d0a32`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:58:08.906575 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-b28886a3-c086-4cc9-91c4-8493cdb743a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:08.999190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1229/2451] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 10 22:58:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:58:09.014864 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0494d7e5-8f39-4acc-b65c-76decce4d116 service cinder] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:09.016666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-0494d7e5-8f39-4acc-b65c-76decce4d116 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:09.083567 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:58:09.124001 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0494d7e5-8f39-4acc-b65c-76decce4d116 service cinder] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:09.129150 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5acc35-340f-4590-98bc-bdb2d3d62851 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e3a76df95a34b9fb9816f22b85d1071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:09.129786 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5acc35-340f-4590-98bc-bdb2d3d62851 service swift] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:09.130061 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5acc35-340f-4590-98bc-bdb2d3d62851 service swift] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:09.130330 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5acc35-340f-4590-98bc-bdb2d3d62851 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:09.140161 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5acc35-340f-4590-98bc-bdb2d3d62851 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:09.141802 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5acc35-340f-4590-98bc-bdb2d3d62851 service swift] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:09.150896 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1223/2452] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 10 22:58:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 22:58:18.797055 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28886a3-c086-4cc9-91c4-8493cdb743a9 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:18.865385 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28886a3-c086-4cc9-91c4-8493cdb743a9 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:18.869917 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc38211-faa3-456c-99e3-8d0107dcd022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:18.870547 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc38211-faa3-456c-99e3-8d0107dcd022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:18.870823 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc38211-faa3-456c-99e3-8d0107dcd022 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:18.871749 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc38211-faa3-456c-99e3-8d0107dcd022 admin admin] PATH_INFO: `/v3/users/11ff134e44b04014afdd13e03ac3ab31` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:18.873089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc38211-faa3-456c-99e3-8d0107dcd022 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11ff134e44b04014afdd13e03ac3ab31)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:18.875030 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc38211-faa3-456c-99e3-8d0107dcd022 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:18.923003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-7bc38211-faa3-456c-99e3-8d0107dcd022 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11ff134e44b04014afdd13e03ac3ab31'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:18.931462 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-7bc38211-faa3-456c-99e3-8d0107dcd022 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11ff134e44b04014afdd13e03ac3ab31'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:18.939574 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-7bc38211-faa3-456c-99e3-8d0107dcd022 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11ff134e44b04014afdd13e03ac3ab31'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:18.940349 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-7bc38211-faa3-456c-99e3-8d0107dcd022 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11ff134e44b04014afdd13e03ac3ab31'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:18.941069 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-7bc38211-faa3-456c-99e3-8d0107dcd022 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11ff134e44b04014afdd13e03ac3ab31'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:18.946457 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:58:18.960301 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1230/2453] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:58:18 2022] DELETE /identity/v3/users/11ff134e44b04014afdd13e03ac3ab31 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:58:18.964925 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c604a-8a8f-43a2-9f91-51362cb4d16f None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:18.965703 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c604a-8a8f-43a2-9f91-51362cb4d16f None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:18.965703 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c604a-8a8f-43a2-9f91-51362cb4d16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:18.985318 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-6d3c604a-8a8f-43a2-9f91-51362cb4d16f None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:58:18.986834 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-6d3c604a-8a8f-43a2-9f91-51362cb4d16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:19.047384 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1224/2454] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:58:19.057252 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc38211-faa3-456c-99e3-8d0107dcd022 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:19.059322 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-7bc38211-faa3-456c-99e3-8d0107dcd022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:19.115691 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc38211-faa3-456c-99e3-8d0107dcd022 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:19.119348 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999fb09-7fcd-4d6e-bfa2-bc422e97f5d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:19.119952 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999fb09-7fcd-4d6e-bfa2-bc422e97f5d9 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:19.120231 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999fb09-7fcd-4d6e-bfa2-bc422e97f5d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:19.120504 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999fb09-7fcd-4d6e-bfa2-bc422e97f5d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:19.129296 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7999fb09-7fcd-4d6e-bfa2-bc422e97f5d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:19.130987 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7999fb09-7fcd-4d6e-bfa2-bc422e97f5d9 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:19.140310 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1231/2455] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:58:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:58:19.409646 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3c604a-8a8f-43a2-9f91-51362cb4d16f None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:19.411068 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-6d3c604a-8a8f-43a2-9f91-51362cb4d16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:19.436811 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3c604a-8a8f-43a2-9f91-51362cb4d16f None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:19.442340 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c8c01-6ea9-484d-8f43-a10bc35af7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:19.442937 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c8c01-6ea9-484d-8f43-a10bc35af7da admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:19.443238 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c8c01-6ea9-484d-8f43-a10bc35af7da admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:19.443538 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c8c01-6ea9-484d-8f43-a10bc35af7da admin admin] PATH_INFO: `/v3/projects/1c3a035c71e2483eb86deb3cc6d3eacc` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:19.446092 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541c8c01-6ea9-484d-8f43-a10bc35af7da admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c3a035c71e2483eb86deb3cc6d3eacc)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:19.446792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541c8c01-6ea9-484d-8f43-a10bc35af7da admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:19.498681 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-541c8c01-6ea9-484d-8f43-a10bc35af7da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c3a035c71e2483eb86deb3cc6d3eacc', 'request_id': 'req-541c8c01-6ea9-484d-8f43-a10bc35af7da'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:19.507619 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-541c8c01-6ea9-484d-8f43-a10bc35af7da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c3a035c71e2483eb86deb3cc6d3eacc', 'request_id': 'req-541c8c01-6ea9-484d-8f43-a10bc35af7da'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:19.514762 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-541c8c01-6ea9-484d-8f43-a10bc35af7da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c3a035c71e2483eb86deb3cc6d3eacc', 'request_id': 'req-541c8c01-6ea9-484d-8f43-a10bc35af7da'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:19.526188 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-541c8c01-6ea9-484d-8f43-a10bc35af7da admin admin] The token cache is being invalidate because project 1c3a035c71e2483eb86deb3cc6d3eacc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:58:19.526472 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-541c8c01-6ea9-484d-8f43-a10bc35af7da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:19.527117 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-541c8c01-6ea9-484d-8f43-a10bc35af7da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:19.530576 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1225/2456] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:58:19 2022] DELETE /identity/v3/projects/1c3a035c71e2483eb86deb3cc6d3eacc => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:58:19.535819 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522bca5-8444-4514-a6de-18c21d8962de None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:19.536108 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522bca5-8444-4514-a6de-18c21d8962de None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:19.536386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522bca5-8444-4514-a6de-18c21d8962de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:19.557018 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-a522bca5-8444-4514-a6de-18c21d8962de None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:58:19.558269 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-a522bca5-8444-4514-a6de-18c21d8962de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:19.619322 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1232/2457] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:58:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:58:19.628630 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bcec5-ad49-41d3-84f4-44f20b11370f None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:19.628916 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bcec5-ad49-41d3-84f4-44f20b11370f None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:19.629190 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bcec5-ad49-41d3-84f4-44f20b11370f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:19.649978 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-540bcec5-ad49-41d3-84f4-44f20b11370f None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:58:19.651205 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-540bcec5-ad49-41d3-84f4-44f20b11370f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:19.660339 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1226/2458] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:58:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:58:19.664974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a522bca5-8444-4514-a6de-18c21d8962de None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:19.674830 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a522bca5-8444-4514-a6de-18c21d8962de None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:19.679411 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff394e-4c07-45e3-80f6-53254e25c296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:19.680007 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff394e-4c07-45e3-80f6-53254e25c296 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:19.680285 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff394e-4c07-45e3-80f6-53254e25c296 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:19.680557 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff394e-4c07-45e3-80f6-53254e25c296 admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:19.681194 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ff394e-4c07-45e3-80f6-53254e25c296 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:58:19.681526 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ff394e-4c07-45e3-80f6-53254e25c296 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:19.683123 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ff394e-4c07-45e3-80f6-53254e25c296 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:19.694831 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1233/2459] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:58:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:58:19.699965 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540bcec5-ad49-41d3-84f4-44f20b11370f None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:19.709625 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540bcec5-ad49-41d3-84f4-44f20b11370f None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:19.714326 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a637f62d-fdba-43cb-a7f6-6441e8e9da53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:19.715556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637f62d-fdba-43cb-a7f6-6441e8e9da53 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:19.715844 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637f62d-fdba-43cb-a7f6-6441e8e9da53 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:19.716119 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637f62d-fdba-43cb-a7f6-6441e8e9da53 admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:19.716741 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a637f62d-fdba-43cb-a7f6-6441e8e9da53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:19.718738 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a637f62d-fdba-43cb-a7f6-6441e8e9da53 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:19.745997 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1227/2460] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:58:19 2022] POST /identity/v3/projects => generated 399 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:58:19.750593 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff394e-4c07-45e3-80f6-53254e25c296 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:19.760732 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff394e-4c07-45e3-80f6-53254e25c296 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:19.817052 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76808b0f-27d3-4990-9c9c-be1e54c29fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:19.817775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76808b0f-27d3-4990-9c9c-be1e54c29fda admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:19.817963 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76808b0f-27d3-4990-9c9c-be1e54c29fda admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:19.818260 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76808b0f-27d3-4990-9c9c-be1e54c29fda admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:19.818893 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76808b0f-27d3-4990-9c9c-be1e54c29fda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:19.820869 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76808b0f-27d3-4990-9c9c-be1e54c29fda admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:19.843080 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1234/2461] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:58:19 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:58:19.847173 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a637f62d-fdba-43cb-a7f6-6441e8e9da53 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:19.859234 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a637f62d-fdba-43cb-a7f6-6441e8e9da53 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:19.864501 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85a785-31a3-4308-ab5f-fd0c084f8a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:19.865106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85a785-31a3-4308-ab5f-fd0c084f8a30 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:19.865419 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85a785-31a3-4308-ab5f-fd0c084f8a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:19.865695 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85a785-31a3-4308-ab5f-fd0c084f8a30 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:19.866341 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab85a785-31a3-4308-ab5f-fd0c084f8a30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:19.867936 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab85a785-31a3-4308-ab5f-fd0c084f8a30 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:19.879080 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1228/2462] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:58:19.883829 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76808b0f-27d3-4990-9c9c-be1e54c29fda admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:19.894737 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76808b0f-27d3-4990-9c9c-be1e54c29fda admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:19.899312 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a83440-a85d-4776-ab88-4a9fc084af66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:19.899908 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a83440-a85d-4776-ab88-4a9fc084af66 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:19.900185 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a83440-a85d-4776-ab88-4a9fc084af66 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:19.900454 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a83440-a85d-4776-ab88-4a9fc084af66 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:19.901087 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a83440-a85d-4776-ab88-4a9fc084af66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:19.902791 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a83440-a85d-4776-ab88-4a9fc084af66 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:19.904092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:58:19.917878 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1235/2463] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:58:19.919112 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85a785-31a3-4308-ab5f-fd0c084f8a30 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:19.924309 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:58:19.930936 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85a785-31a3-4308-ab5f-fd0c084f8a30 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:19.937056 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533b257-cc5d-4e69-8b1b-520b20cd3a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:19.937743 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533b257-cc5d-4e69-8b1b-520b20cd3a49 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:19.938027 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533b257-cc5d-4e69-8b1b-520b20cd3a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:19.938302 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533b257-cc5d-4e69-8b1b-520b20cd3a49 admin admin] PATH_INFO: `/v3/projects/c6db7eb6d3924d40a28c956edf4b8d98/users/42e2fb8ec43645a7b771d92dc7ad04e2/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:19.953786 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533b257-cc5d-4e69-8b1b-520b20cd3a49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6db7eb6d3924d40a28c956edf4b8d98, user_id=42e2fb8ec43645a7b771d92dc7ad04e2, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:19.956450 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533b257-cc5d-4e69-8b1b-520b20cd3a49 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:19.970158 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1229/2464] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:58:19 2022] PUT /identity/v3/projects/c6db7eb6d3924d40a28c956edf4b8d98/users/42e2fb8ec43645a7b771d92dc7ad04e2/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:58:19.981095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ef062-78dd-4a1a-a6c7-947502c3d760 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:19.981934 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ef062-78dd-4a1a-a6c7-947502c3d760 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:19.981934 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ef062-78dd-4a1a-a6c7-947502c3d760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:20.009297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-005ef062-78dd-4a1a-a6c7-947502c3d760 None None] MFA Rules not processed for user `42e2fb8ec43645a7b771d92dc7ad04e2`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:58:20.010525 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-005ef062-78dd-4a1a-a6c7-947502c3d760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:20.074465 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1236/2465] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:58:19 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:58:20.086085 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533b257-cc5d-4e69-8b1b-520b20cd3a49 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:20.087773 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-2533b257-cc5d-4e69-8b1b-520b20cd3a49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:20.149209 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533b257-cc5d-4e69-8b1b-520b20cd3a49 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:20.153424 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b430ae3c-807e-414b-981e-8d3cc27e5892 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:20.154161 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430ae3c-807e-414b-981e-8d3cc27e5892 service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:20.154474 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430ae3c-807e-414b-981e-8d3cc27e5892 service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:20.154880 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430ae3c-807e-414b-981e-8d3cc27e5892 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:20.164509 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b430ae3c-807e-414b-981e-8d3cc27e5892 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:20.165993 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b430ae3c-807e-414b-981e-8d3cc27e5892 service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:20.175674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1230/2466] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:58:20 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:58:30.071372 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=34 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:58:30.171773 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=35,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:58:30.329384 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ef062-78dd-4a1a-a6c7-947502c3d760 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:30.391879 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ef062-78dd-4a1a-a6c7-947502c3d760 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:30.396349 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b30c588-60e9-4fc7-8e91-74c889ec0f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:30.396976 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30c588-60e9-4fc7-8e91-74c889ec0f6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:30.397260 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30c588-60e9-4fc7-8e91-74c889ec0f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:30.397562 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30c588-60e9-4fc7-8e91-74c889ec0f6b admin admin] PATH_INFO: `/v3/users/42e2fb8ec43645a7b771d92dc7ad04e2` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:30.398673 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b30c588-60e9-4fc7-8e91-74c889ec0f6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=42e2fb8ec43645a7b771d92dc7ad04e2)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:30.400595 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b30c588-60e9-4fc7-8e91-74c889ec0f6b admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:30.446567 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-8b30c588-60e9-4fc7-8e91-74c889ec0f6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42e2fb8ec43645a7b771d92dc7ad04e2'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:30.454302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-8b30c588-60e9-4fc7-8e91-74c889ec0f6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42e2fb8ec43645a7b771d92dc7ad04e2'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:30.461396 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-8b30c588-60e9-4fc7-8e91-74c889ec0f6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42e2fb8ec43645a7b771d92dc7ad04e2'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:30.462055 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-8b30c588-60e9-4fc7-8e91-74c889ec0f6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42e2fb8ec43645a7b771d92dc7ad04e2'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:30.462670 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-8b30c588-60e9-4fc7-8e91-74c889ec0f6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42e2fb8ec43645a7b771d92dc7ad04e2'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:30.481496 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1237/2467] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:58:30 2022] DELETE /identity/v3/users/42e2fb8ec43645a7b771d92dc7ad04e2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:58:30.485983 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b14b4e-e6a0-4581-871a-6162dd4a17a0 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:30.486297 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b14b4e-e6a0-4581-871a-6162dd4a17a0 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:30.486575 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b14b4e-e6a0-4581-871a-6162dd4a17a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:30.506184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-58b14b4e-e6a0-4581-871a-6162dd4a17a0 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:58:30.507389 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-58b14b4e-e6a0-4581-871a-6162dd4a17a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:30.566072 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1231/2468] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:58:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:58:30.576429 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b30c588-60e9-4fc7-8e91-74c889ec0f6b admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:30.577933 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-8b30c588-60e9-4fc7-8e91-74c889ec0f6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:30.634703 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b30c588-60e9-4fc7-8e91-74c889ec0f6b admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:30.639297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc61ac7-b28d-4393-8de5-4b54f2836d40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:30.639906 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc61ac7-b28d-4393-8de5-4b54f2836d40 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:30.640195 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc61ac7-b28d-4393-8de5-4b54f2836d40 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:30.640469 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc61ac7-b28d-4393-8de5-4b54f2836d40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:30.649593 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc61ac7-b28d-4393-8de5-4b54f2836d40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:30.651035 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc61ac7-b28d-4393-8de5-4b54f2836d40 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:30.661591 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1238/2469] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:58:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:58:30.931016 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b14b4e-e6a0-4581-871a-6162dd4a17a0 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:30.932470 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-58b14b4e-e6a0-4581-871a-6162dd4a17a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:30.949333 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b14b4e-e6a0-4581-871a-6162dd4a17a0 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:30.954071 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fcd830-4464-4afb-8752-cb154f288d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:30.954862 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fcd830-4464-4afb-8752-cb154f288d65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:30.955022 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fcd830-4464-4afb-8752-cb154f288d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:30.955300 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fcd830-4464-4afb-8752-cb154f288d65 admin admin] PATH_INFO: `/v3/projects/c6db7eb6d3924d40a28c956edf4b8d98` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:30.956512 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fcd830-4464-4afb-8752-cb154f288d65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6db7eb6d3924d40a28c956edf4b8d98)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:30.958487 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fcd830-4464-4afb-8752-cb154f288d65 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:31.005611 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-41fcd830-4464-4afb-8752-cb154f288d65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6db7eb6d3924d40a28c956edf4b8d98', 'request_id': 'req-41fcd830-4464-4afb-8752-cb154f288d65'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:31.012965 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-41fcd830-4464-4afb-8752-cb154f288d65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6db7eb6d3924d40a28c956edf4b8d98', 'request_id': 'req-41fcd830-4464-4afb-8752-cb154f288d65'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:31.018827 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-41fcd830-4464-4afb-8752-cb154f288d65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6db7eb6d3924d40a28c956edf4b8d98', 'request_id': 'req-41fcd830-4464-4afb-8752-cb154f288d65'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:31.030075 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-41fcd830-4464-4afb-8752-cb154f288d65 admin admin] The token cache is being invalidate because project c6db7eb6d3924d40a28c956edf4b8d98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:58:31.030356 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-41fcd830-4464-4afb-8752-cb154f288d65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:31.030974 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-41fcd830-4464-4afb-8752-cb154f288d65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:31.032447 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1232/2470] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:58:30 2022] DELETE /identity/v3/projects/c6db7eb6d3924d40a28c956edf4b8d98 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:58:35.574596 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d73f1f-d465-4683-99f3-82ab43c1d587 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:35.575172 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d73f1f-d465-4683-99f3-82ab43c1d587 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:35.575172 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d73f1f-d465-4683-99f3-82ab43c1d587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:35.595402 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-75d73f1f-d465-4683-99f3-82ab43c1d587 None None] MFA Rules not processed for user `aff5e804c64345b298c745287a1d0a32`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:58:35.596602 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-75d73f1f-d465-4683-99f3-82ab43c1d587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:35.613537 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:58:35.655860 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1239/2471] 10.222.0.76 () {58 vars in 967 bytes} [Wed Aug 10 22:58:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:58:35.676597 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fcd830-4464-4afb-8752-cb154f288d65 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:35.678030 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-41fcd830-4464-4afb-8752-cb154f288d65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:35.747197 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fcd830-4464-4afb-8752-cb154f288d65 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:35.752068 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d311e0-8240-47a6-98d1-f224a6102789 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e3a76df95a34b9fb9816f22b85d1071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:35.752670 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d311e0-8240-47a6-98d1-f224a6102789 service swift] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:35.752945 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d311e0-8240-47a6-98d1-f224a6102789 service swift] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:35.753493 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d311e0-8240-47a6-98d1-f224a6102789 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:35.762322 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d311e0-8240-47a6-98d1-f224a6102789 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:35.763901 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d311e0-8240-47a6-98d1-f224a6102789 service swift] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:35.771962 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1233/2472] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 10 22:58:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 22:58:38.229112 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d73f1f-d465-4683-99f3-82ab43c1d587 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:38.231505 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-75d73f1f-d465-4683-99f3-82ab43c1d587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:38.313620 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d73f1f-d465-4683-99f3-82ab43c1d587 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:38.319102 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae68301c-dcad-47f1-825b-e915e2582afa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:38.319709 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae68301c-dcad-47f1-825b-e915e2582afa service nova] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:38.319999 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae68301c-dcad-47f1-825b-e915e2582afa service nova] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:38.320299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae68301c-dcad-47f1-825b-e915e2582afa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:38.322049 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-ae68301c-dcad-47f1-825b-e915e2582afa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:38.401742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae68301c-dcad-47f1-825b-e915e2582afa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:38.403486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae68301c-dcad-47f1-825b-e915e2582afa service nova] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:38.422346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1240/2473] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:58:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:58:45.768771 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:58:46.788805 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d311e0-8240-47a6-98d1-f224a6102789 service swift] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:46.790559 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-99d311e0-8240-47a6-98d1-f224a6102789 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:46.854211 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d311e0-8240-47a6-98d1-f224a6102789 service swift] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:46.858611 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81081da3-eb1c-4889-8171-538f4b8982b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:46.859217 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81081da3-eb1c-4889-8171-538f4b8982b1 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:46.859492 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81081da3-eb1c-4889-8171-538f4b8982b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:46.859760 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81081da3-eb1c-4889-8171-538f4b8982b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:46.861538 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-81081da3-eb1c-4889-8171-538f4b8982b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:46.927131 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81081da3-eb1c-4889-8171-538f4b8982b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:46.928552 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81081da3-eb1c-4889-8171-538f4b8982b1 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:46.938368 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1234/2474] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:58:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:58:48.419612 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=84 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:58:53.651618 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae68301c-dcad-47f1-825b-e915e2582afa service nova] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:53.654717 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-ae68301c-dcad-47f1-825b-e915e2582afa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:53.677787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae68301c-dcad-47f1-825b-e915e2582afa service nova] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:53.683212 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27f81f-fc1a-4e66-9eca-8315bbb0b2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:53.683908 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27f81f-fc1a-4e66-9eca-8315bbb0b2f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:53.684184 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27f81f-fc1a-4e66-9eca-8315bbb0b2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:53.684459 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27f81f-fc1a-4e66-9eca-8315bbb0b2f9 admin admin] PATH_INFO: `/v3/users/f5fe2bdd1ea54f8cba4d100b26c18464` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:53.685668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf27f81f-fc1a-4e66-9eca-8315bbb0b2f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5fe2bdd1ea54f8cba4d100b26c18464)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:53.687599 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf27f81f-fc1a-4e66-9eca-8315bbb0b2f9 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:53.744285 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-cf27f81f-fc1a-4e66-9eca-8315bbb0b2f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5fe2bdd1ea54f8cba4d100b26c18464'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:53.753098 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-cf27f81f-fc1a-4e66-9eca-8315bbb0b2f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5fe2bdd1ea54f8cba4d100b26c18464'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:53.759958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-cf27f81f-fc1a-4e66-9eca-8315bbb0b2f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5fe2bdd1ea54f8cba4d100b26c18464'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:53.761374 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-cf27f81f-fc1a-4e66-9eca-8315bbb0b2f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5fe2bdd1ea54f8cba4d100b26c18464'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:53.762046 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-cf27f81f-fc1a-4e66-9eca-8315bbb0b2f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5fe2bdd1ea54f8cba4d100b26c18464'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:53.779586 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1241/2475] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:58:53 2022] DELETE /identity/v3/users/f5fe2bdd1ea54f8cba4d100b26c18464 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:58:53.956877 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81081da3-eb1c-4889-8171-538f4b8982b1 service neutron] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:53.968200 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81081da3-eb1c-4889-8171-538f4b8982b1 service neutron] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:54.028239 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe64635-4985-4574-be64-bead7994563f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:54.028874 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe64635-4985-4574-be64-bead7994563f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:54.029153 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe64635-4985-4574-be64-bead7994563f admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:54.029561 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe64635-4985-4574-be64-bead7994563f admin admin] PATH_INFO: `/v3/projects/2df73c10b6654d47a921aa6866d81526` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:54.030985 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe64635-4985-4574-be64-bead7994563f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2df73c10b6654d47a921aa6866d81526)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:54.032922 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe64635-4985-4574-be64-bead7994563f admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:54.078989 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-8fe64635-4985-4574-be64-bead7994563f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2df73c10b6654d47a921aa6866d81526', 'request_id': 'req-8fe64635-4985-4574-be64-bead7994563f'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:54.081370 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:58:54.086600 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-8fe64635-4985-4574-be64-bead7994563f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2df73c10b6654d47a921aa6866d81526', 'request_id': 'req-8fe64635-4985-4574-be64-bead7994563f'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:54.093452 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-8fe64635-4985-4574-be64-bead7994563f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2df73c10b6654d47a921aa6866d81526', 'request_id': 'req-8fe64635-4985-4574-be64-bead7994563f'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:54.104869 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-8fe64635-4985-4574-be64-bead7994563f admin admin] The token cache is being invalidate because project 2df73c10b6654d47a921aa6866d81526 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:58:54.105135 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-8fe64635-4985-4574-be64-bead7994563f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:54.105814 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-8fe64635-4985-4574-be64-bead7994563f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:54.107298 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1235/2476] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:58:53 2022] DELETE /identity/v3/projects/2df73c10b6654d47a921aa6866d81526 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:58:54.111311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27f81f-fc1a-4e66-9eca-8315bbb0b2f9 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:54.112736 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-cf27f81f-fc1a-4e66-9eca-8315bbb0b2f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:54.127204 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27f81f-fc1a-4e66-9eca-8315bbb0b2f9 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:54.183605 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd02cc4-120c-420c-92ca-edea718bbf94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:54.184239 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd02cc4-120c-420c-92ca-edea718bbf94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:54.184520 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd02cc4-120c-420c-92ca-edea718bbf94 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:54.184794 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd02cc4-120c-420c-92ca-edea718bbf94 admin admin] PATH_INFO: `/v3/users/1df55d4c35794a33a6133c19412da7c6` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:54.185990 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd02cc4-120c-420c-92ca-edea718bbf94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1df55d4c35794a33a6133c19412da7c6)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:54.187924 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd02cc4-120c-420c-92ca-edea718bbf94 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:54.232787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-9fd02cc4-120c-420c-92ca-edea718bbf94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1df55d4c35794a33a6133c19412da7c6'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:54.240277 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-9fd02cc4-120c-420c-92ca-edea718bbf94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1df55d4c35794a33a6133c19412da7c6'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:54.247883 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-9fd02cc4-120c-420c-92ca-edea718bbf94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1df55d4c35794a33a6133c19412da7c6'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:54.248550 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-9fd02cc4-120c-420c-92ca-edea718bbf94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1df55d4c35794a33a6133c19412da7c6'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:54.249146 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-9fd02cc4-120c-420c-92ca-edea718bbf94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1df55d4c35794a33a6133c19412da7c6'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:54.257436 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:58:54.269350 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1242/2477] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:58:54 2022] DELETE /identity/v3/users/1df55d4c35794a33a6133c19412da7c6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:58:54.442132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe64635-4985-4574-be64-bead7994563f admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:58:54.443684 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-8fe64635-4985-4574-be64-bead7994563f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:58:54.458005 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe64635-4985-4574-be64-bead7994563f admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:58:54.515620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7c073b-28ac-4fa4-924a-69a59ce1d8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:58:54.516256 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c073b-28ac-4fa4-924a-69a59ce1d8f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:58:54.516531 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c073b-28ac-4fa4-924a-69a59ce1d8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:58:54.516798 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c073b-28ac-4fa4-924a-69a59ce1d8f5 admin admin] PATH_INFO: `/v3/projects/bc5da3cccdac4bb8b3a7ebb989d431b9` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:58:54.518244 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7c073b-28ac-4fa4-924a-69a59ce1d8f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc5da3cccdac4bb8b3a7ebb989d431b9)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:58:54.520232 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7c073b-28ac-4fa4-924a-69a59ce1d8f5 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:58:54.567525 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-da7c073b-28ac-4fa4-924a-69a59ce1d8f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc5da3cccdac4bb8b3a7ebb989d431b9', 'request_id': 'req-da7c073b-28ac-4fa4-924a-69a59ce1d8f5'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:54.575316 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-da7c073b-28ac-4fa4-924a-69a59ce1d8f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc5da3cccdac4bb8b3a7ebb989d431b9', 'request_id': 'req-da7c073b-28ac-4fa4-924a-69a59ce1d8f5'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:54.580986 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-da7c073b-28ac-4fa4-924a-69a59ce1d8f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc5da3cccdac4bb8b3a7ebb989d431b9', 'request_id': 'req-da7c073b-28ac-4fa4-924a-69a59ce1d8f5'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:54.593327 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-da7c073b-28ac-4fa4-924a-69a59ce1d8f5 admin admin] The token cache is being invalidate because project bc5da3cccdac4bb8b3a7ebb989d431b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:58:54.593834 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-da7c073b-28ac-4fa4-924a-69a59ce1d8f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:54.594475 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-da7c073b-28ac-4fa4-924a-69a59ce1d8f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:58:54.595953 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1236/2478] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:58:54 2022] DELETE /identity/v3/projects/bc5da3cccdac4bb8b3a7ebb989d431b9 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:04.267371 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:04.590230 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:DELETE=10,keystone:SELECT=57,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:10.193865 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd02cc4-120c-420c-92ca-edea718bbf94 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:10.195305 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-9fd02cc4-120c-420c-92ca-edea718bbf94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:10.253355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd02cc4-120c-420c-92ca-edea718bbf94 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:10.300766 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61220d91-64f6-4caf-8103-596e2cb43d10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:10.301406 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61220d91-64f6-4caf-8103-596e2cb43d10 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:10.301683 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61220d91-64f6-4caf-8103-596e2cb43d10 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:10.301955 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61220d91-64f6-4caf-8103-596e2cb43d10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:10.303747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-61220d91-64f6-4caf-8103-596e2cb43d10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:10.365514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61220d91-64f6-4caf-8103-596e2cb43d10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:10.366864 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61220d91-64f6-4caf-8103-596e2cb43d10 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:10.376864 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1243/2479] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:59:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:59:13.256805 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7c073b-28ac-4fa4-924a-69a59ce1d8f5 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:13.258493 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-da7c073b-28ac-4fa4-924a-69a59ce1d8f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:13.353659 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61220d91-64f6-4caf-8103-596e2cb43d10 service neutron] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:13.354103 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-61220d91-64f6-4caf-8103-596e2cb43d10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:13.361778 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7c073b-28ac-4fa4-924a-69a59ce1d8f5 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:13.369216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cdf91f-940b-4ba5-af14-b8fbdd58a364 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f46dee0f73e49eab54c11556e3b86bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:13.370186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cdf91f-940b-4ba5-af14-b8fbdd58a364 service placement] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:13.370470 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cdf91f-940b-4ba5-af14-b8fbdd58a364 service placement] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:13.370745 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cdf91f-940b-4ba5-af14-b8fbdd58a364 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:13.372537 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-b9cdf91f-940b-4ba5-af14-b8fbdd58a364 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:13.373106 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61220d91-64f6-4caf-8103-596e2cb43d10 service neutron] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:13.379021 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a3159-b2b6-4eeb-a6f4-24b341ffa9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:13.379695 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a3159-b2b6-4eeb-a6f4-24b341ffa9ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:13.380034 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a3159-b2b6-4eeb-a6f4-24b341ffa9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:13.380244 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a3159-b2b6-4eeb-a6f4-24b341ffa9ec admin admin] PATH_INFO: `/v3/users/405738ed219b4fdc8e3c07706a13e5a2` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:13.381428 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a3159-b2b6-4eeb-a6f4-24b341ffa9ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=405738ed219b4fdc8e3c07706a13e5a2)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:13.383662 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a3159-b2b6-4eeb-a6f4-24b341ffa9ec admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:13.387559 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cdf91f-940b-4ba5-af14-b8fbdd58a364 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:13.389155 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cdf91f-940b-4ba5-af14-b8fbdd58a364 service placement] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:13.401354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1237/2480] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:59:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:59:13.444053 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-0b6a3159-b2b6-4eeb-a6f4-24b341ffa9ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '405738ed219b4fdc8e3c07706a13e5a2'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:13.450920 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-0b6a3159-b2b6-4eeb-a6f4-24b341ffa9ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '405738ed219b4fdc8e3c07706a13e5a2'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:13.457017 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:13.459198 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-0b6a3159-b2b6-4eeb-a6f4-24b341ffa9ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '405738ed219b4fdc8e3c07706a13e5a2'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:13.460383 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-0b6a3159-b2b6-4eeb-a6f4-24b341ffa9ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '405738ed219b4fdc8e3c07706a13e5a2'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:13.461024 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-0b6a3159-b2b6-4eeb-a6f4-24b341ffa9ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '405738ed219b4fdc8e3c07706a13e5a2'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:13.481544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1244/2481] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:59:13 2022] DELETE /identity/v3/users/405738ed219b4fdc8e3c07706a13e5a2 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:13.653498 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cdf91f-940b-4ba5-af14-b8fbdd58a364 service placement] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:13.665969 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cdf91f-940b-4ba5-af14-b8fbdd58a364 service placement] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:13.726821 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd8c7a-4830-4f86-87bc-7cf1c78f5bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:13.728148 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd8c7a-4830-4f86-87bc-7cf1c78f5bb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:13.728505 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd8c7a-4830-4f86-87bc-7cf1c78f5bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:13.728811 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd8c7a-4830-4f86-87bc-7cf1c78f5bb5 admin admin] PATH_INFO: `/v3/projects/b81a25303ced4ea4bf5351ec42192c13` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:13.729978 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cd8c7a-4830-4f86-87bc-7cf1c78f5bb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b81a25303ced4ea4bf5351ec42192c13)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:13.731869 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cd8c7a-4830-4f86-87bc-7cf1c78f5bb5 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:13.778510 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-32cd8c7a-4830-4f86-87bc-7cf1c78f5bb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b81a25303ced4ea4bf5351ec42192c13', 'request_id': 'req-32cd8c7a-4830-4f86-87bc-7cf1c78f5bb5'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:13.785847 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-32cd8c7a-4830-4f86-87bc-7cf1c78f5bb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b81a25303ced4ea4bf5351ec42192c13', 'request_id': 'req-32cd8c7a-4830-4f86-87bc-7cf1c78f5bb5'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:13.791945 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-32cd8c7a-4830-4f86-87bc-7cf1c78f5bb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b81a25303ced4ea4bf5351ec42192c13', 'request_id': 'req-32cd8c7a-4830-4f86-87bc-7cf1c78f5bb5'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:13.802793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-32cd8c7a-4830-4f86-87bc-7cf1c78f5bb5 admin admin] The token cache is being invalidate because project b81a25303ced4ea4bf5351ec42192c13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:59:13.803076 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-32cd8c7a-4830-4f86-87bc-7cf1c78f5bb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:13.803718 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-32cd8c7a-4830-4f86-87bc-7cf1c78f5bb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:13.805921 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1238/2482] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:59:13 2022] DELETE /identity/v3/projects/b81a25303ced4ea4bf5351ec42192c13 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:13.811673 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d7777-b194-4e33-97d2-831d82a4758b None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:13.811960 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d7777-b194-4e33-97d2-831d82a4758b None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:13.812232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d7777-b194-4e33-97d2-831d82a4758b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:13.836271 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-685d7777-b194-4e33-97d2-831d82a4758b None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:59:13.838302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-685d7777-b194-4e33-97d2-831d82a4758b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:13.901548 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1245/2483] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:59:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:59:13.908997 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62703a80-8cd8-477d-9732-f37aeb21a8b0 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:13.909302 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62703a80-8cd8-477d-9732-f37aeb21a8b0 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:13.909593 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62703a80-8cd8-477d-9732-f37aeb21a8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:13.929859 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-62703a80-8cd8-477d-9732-f37aeb21a8b0 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:59:13.931196 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-62703a80-8cd8-477d-9732-f37aeb21a8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:13.939356 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1239/2484] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:59:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:59:13.943481 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d7777-b194-4e33-97d2-831d82a4758b None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:13.953517 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d7777-b194-4e33-97d2-831d82a4758b None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:13.958215 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9848e1f5-291c-44c3-a906-c026e919c3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:13.958824 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848e1f5-291c-44c3-a906-c026e919c3a5 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:13.959106 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848e1f5-291c-44c3-a906-c026e919c3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:13.959381 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848e1f5-291c-44c3-a906-c026e919c3a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:13.960025 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9848e1f5-291c-44c3-a906-c026e919c3a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:59:13.960327 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9848e1f5-291c-44c3-a906-c026e919c3a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:13.962564 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9848e1f5-291c-44c3-a906-c026e919c3a5 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:13.973990 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1246/2485] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:59:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:59:13.978335 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62703a80-8cd8-477d-9732-f37aeb21a8b0 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:13.987452 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62703a80-8cd8-477d-9732-f37aeb21a8b0 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:13.992133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b53ea37-b28f-4bce-9ebc-09bd5a8e3cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:13.993240 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53ea37-b28f-4bce-9ebc-09bd5a8e3cbe admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:13.993554 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53ea37-b28f-4bce-9ebc-09bd5a8e3cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:13.993825 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53ea37-b28f-4bce-9ebc-09bd5a8e3cbe admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:13.994438 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b53ea37-b28f-4bce-9ebc-09bd5a8e3cbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:13.996347 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b53ea37-b28f-4bce-9ebc-09bd5a8e3cbe admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:14.007620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:14.025139 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1240/2486] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:59:13 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:59:14.029384 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9848e1f5-291c-44c3-a906-c026e919c3a5 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:14.039275 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9848e1f5-291c-44c3-a906-c026e919c3a5 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:14.095001 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d674a91-1ac3-4819-b224-ddc720d89f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:14.095601 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d674a91-1ac3-4819-b224-ddc720d89f93 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:14.095876 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d674a91-1ac3-4819-b224-ddc720d89f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:14.096145 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d674a91-1ac3-4819-b224-ddc720d89f93 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:14.096752 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d674a91-1ac3-4819-b224-ddc720d89f93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:14.098720 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d674a91-1ac3-4819-b224-ddc720d89f93 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:14.119922 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1247/2487] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:59:14 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:59:14.124106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b53ea37-b28f-4bce-9ebc-09bd5a8e3cbe admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:14.135661 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b53ea37-b28f-4bce-9ebc-09bd5a8e3cbe admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:14.140484 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee2e87e-96f2-4f59-8eed-6b07fe3f0a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:14.141083 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2e87e-96f2-4f59-8eed-6b07fe3f0a4e admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:14.141402 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2e87e-96f2-4f59-8eed-6b07fe3f0a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:14.141678 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2e87e-96f2-4f59-8eed-6b07fe3f0a4e admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:14.142316 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee2e87e-96f2-4f59-8eed-6b07fe3f0a4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:14.143920 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee2e87e-96f2-4f59-8eed-6b07fe3f0a4e admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:14.154172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1241/2488] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:59:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:59:14.158193 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d674a91-1ac3-4819-b224-ddc720d89f93 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:14.168487 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d674a91-1ac3-4819-b224-ddc720d89f93 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:14.173302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ccaac4-6b5b-4acb-9413-f7701056fc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:14.173907 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ccaac4-6b5b-4acb-9413-f7701056fc66 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:14.174187 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ccaac4-6b5b-4acb-9413-f7701056fc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:14.174461 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ccaac4-6b5b-4acb-9413-f7701056fc66 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:14.175107 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ccaac4-6b5b-4acb-9413-f7701056fc66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:14.176698 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ccaac4-6b5b-4acb-9413-f7701056fc66 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:14.189014 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1248/2489] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:59:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:59:14.193192 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee2e87e-96f2-4f59-8eed-6b07fe3f0a4e admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:14.204112 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee2e87e-96f2-4f59-8eed-6b07fe3f0a4e admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:14.208719 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d136e06-b20d-4a07-8c98-4741fd84b5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:14.209385 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d136e06-b20d-4a07-8c98-4741fd84b5ba admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:14.209663 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d136e06-b20d-4a07-8c98-4741fd84b5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:14.209934 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d136e06-b20d-4a07-8c98-4741fd84b5ba admin admin] PATH_INFO: `/v3/projects/762d90c0aed04f1583a5f96c13f2c86e/users/be5c322583694f2aa32df288bd0ec14d/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:14.224977 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d136e06-b20d-4a07-8c98-4741fd84b5ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=762d90c0aed04f1583a5f96c13f2c86e, user_id=be5c322583694f2aa32df288bd0ec14d, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:14.228205 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d136e06-b20d-4a07-8c98-4741fd84b5ba admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:14.240960 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1242/2490] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:59:14 2022] PUT /identity/v3/projects/762d90c0aed04f1583a5f96c13f2c86e/users/be5c322583694f2aa32df288bd0ec14d/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:14.253817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab929ff3-d6cb-45f5-bcae-01b48aab2411 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:14.254107 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab929ff3-d6cb-45f5-bcae-01b48aab2411 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:14.254386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab929ff3-d6cb-45f5-bcae-01b48aab2411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:14.281592 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-ab929ff3-d6cb-45f5-bcae-01b48aab2411 None None] MFA Rules not processed for user `be5c322583694f2aa32df288bd0ec14d`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:59:14.282806 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-ab929ff3-d6cb-45f5-bcae-01b48aab2411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:14.314864 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:14.347087 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1249/2491] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:59:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:59:14.358927 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d136e06-b20d-4a07-8c98-4741fd84b5ba admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:14.360839 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-0d136e06-b20d-4a07-8c98-4741fd84b5ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:14.436289 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d136e06-b20d-4a07-8c98-4741fd84b5ba admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:14.441951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c63d7e-b4bf-49d7-8999-8e2476c139f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:14.442585 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c63d7e-b4bf-49d7-8999-8e2476c139f9 service nova] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:14.442867 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c63d7e-b4bf-49d7-8999-8e2476c139f9 service nova] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:14.443138 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c63d7e-b4bf-49d7-8999-8e2476c139f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:14.451184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c63d7e-b4bf-49d7-8999-8e2476c139f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:14.452867 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c63d7e-b4bf-49d7-8999-8e2476c139f9 service nova] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:14.461805 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1243/2492] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:59:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:59:17.156001 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab929ff3-d6cb-45f5-bcae-01b48aab2411 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:17.216552 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab929ff3-d6cb-45f5-bcae-01b48aab2411 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:17.221007 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8d370e-43a5-47e9-9bf0-299579c6bffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:17.221672 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d370e-43a5-47e9-9bf0-299579c6bffa admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:17.221945 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d370e-43a5-47e9-9bf0-299579c6bffa admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:17.222211 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d370e-43a5-47e9-9bf0-299579c6bffa admin admin] PATH_INFO: `/v3/users/be5c322583694f2aa32df288bd0ec14d` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:17.223317 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8d370e-43a5-47e9-9bf0-299579c6bffa admin admin] RBAC: Authorizing `identity:delete_user(user_id=be5c322583694f2aa32df288bd0ec14d)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:17.225234 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8d370e-43a5-47e9-9bf0-299579c6bffa admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:17.272943 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-6e8d370e-43a5-47e9-9bf0-299579c6bffa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be5c322583694f2aa32df288bd0ec14d'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:17.280351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-6e8d370e-43a5-47e9-9bf0-299579c6bffa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be5c322583694f2aa32df288bd0ec14d'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:17.287269 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-6e8d370e-43a5-47e9-9bf0-299579c6bffa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be5c322583694f2aa32df288bd0ec14d'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:17.287909 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-6e8d370e-43a5-47e9-9bf0-299579c6bffa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be5c322583694f2aa32df288bd0ec14d'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:17.288534 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-6e8d370e-43a5-47e9-9bf0-299579c6bffa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be5c322583694f2aa32df288bd0ec14d'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:17.307600 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1250/2493] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:59:17 2022] DELETE /identity/v3/users/be5c322583694f2aa32df288bd0ec14d => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:17.312125 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816f71c-cb63-4c8b-b03a-26b12be23db6 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:17.312597 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816f71c-cb63-4c8b-b03a-26b12be23db6 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:17.312705 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816f71c-cb63-4c8b-b03a-26b12be23db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:17.331560 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-1816f71c-cb63-4c8b-b03a-26b12be23db6 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:59:17.332764 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-1816f71c-cb63-4c8b-b03a-26b12be23db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:17.398267 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1244/2494] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:59:17.408270 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8d370e-43a5-47e9-9bf0-299579c6bffa admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:17.409723 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-6e8d370e-43a5-47e9-9bf0-299579c6bffa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:17.467451 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8d370e-43a5-47e9-9bf0-299579c6bffa admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:17.471126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f1c59-8a7c-4fa6-8e55-9d59627fa4d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:17.471727 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f1c59-8a7c-4fa6-8e55-9d59627fa4d5 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:17.472007 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f1c59-8a7c-4fa6-8e55-9d59627fa4d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:17.472278 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f1c59-8a7c-4fa6-8e55-9d59627fa4d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:17.479562 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:17.482077 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234f1c59-8a7c-4fa6-8e55-9d59627fa4d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:17.484662 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234f1c59-8a7c-4fa6-8e55-9d59627fa4d5 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:17.495925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1251/2495] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:59:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:59:17.786757 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816f71c-cb63-4c8b-b03a-26b12be23db6 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:17.788348 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-1816f71c-cb63-4c8b-b03a-26b12be23db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:17.802013 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816f71c-cb63-4c8b-b03a-26b12be23db6 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:17.807350 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c0b3d-9f3e-429d-a58f-60ed208eae62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:17.808001 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c0b3d-9f3e-429d-a58f-60ed208eae62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:17.808321 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c0b3d-9f3e-429d-a58f-60ed208eae62 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:17.808650 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c0b3d-9f3e-429d-a58f-60ed208eae62 admin admin] PATH_INFO: `/v3/projects/762d90c0aed04f1583a5f96c13f2c86e` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:17.809831 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c0b3d-9f3e-429d-a58f-60ed208eae62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=762d90c0aed04f1583a5f96c13f2c86e)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:17.811800 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c0b3d-9f3e-429d-a58f-60ed208eae62 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:17.857442 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:17.867726 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-2a8c0b3d-9f3e-429d-a58f-60ed208eae62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '762d90c0aed04f1583a5f96c13f2c86e', 'request_id': 'req-2a8c0b3d-9f3e-429d-a58f-60ed208eae62'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:17.875653 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-2a8c0b3d-9f3e-429d-a58f-60ed208eae62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '762d90c0aed04f1583a5f96c13f2c86e', 'request_id': 'req-2a8c0b3d-9f3e-429d-a58f-60ed208eae62'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:17.881109 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-2a8c0b3d-9f3e-429d-a58f-60ed208eae62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '762d90c0aed04f1583a5f96c13f2c86e', 'request_id': 'req-2a8c0b3d-9f3e-429d-a58f-60ed208eae62'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:17.892153 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-2a8c0b3d-9f3e-429d-a58f-60ed208eae62 admin admin] The token cache is being invalidate because project 762d90c0aed04f1583a5f96c13f2c86e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:59:17.892440 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-2a8c0b3d-9f3e-429d-a58f-60ed208eae62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:17.893070 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-2a8c0b3d-9f3e-429d-a58f-60ed208eae62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:17.894581 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1245/2496] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:59:17 2022] DELETE /identity/v3/projects/762d90c0aed04f1583a5f96c13f2c86e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:17.900233 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336c5e6-0ebe-4be5-904b-dae942b394bf None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:17.900462 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336c5e6-0ebe-4be5-904b-dae942b394bf None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:17.900739 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336c5e6-0ebe-4be5-904b-dae942b394bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:17.920264 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-d336c5e6-0ebe-4be5-904b-dae942b394bf None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:59:17.921495 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-d336c5e6-0ebe-4be5-904b-dae942b394bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:17.979793 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1252/2497] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:59:17.988026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7bce9-0821-4a9b-b8c5-423518a27ec1 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:17.988308 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7bce9-0821-4a9b-b8c5-423518a27ec1 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:17.988582 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7bce9-0821-4a9b-b8c5-423518a27ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:18.009292 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-dab7bce9-0821-4a9b-b8c5-423518a27ec1 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:59:18.010520 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-dab7bce9-0821-4a9b-b8c5-423518a27ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:18.018369 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1246/2498] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:59:18.022573 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336c5e6-0ebe-4be5-904b-dae942b394bf None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:18.032739 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336c5e6-0ebe-4be5-904b-dae942b394bf None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:18.037256 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f228070f-aecb-4e29-82b2-c124947ce205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:18.037889 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f228070f-aecb-4e29-82b2-c124947ce205 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:18.038165 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f228070f-aecb-4e29-82b2-c124947ce205 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:18.038437 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f228070f-aecb-4e29-82b2-c124947ce205 admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:18.039079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f228070f-aecb-4e29-82b2-c124947ce205 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:59:18.039379 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f228070f-aecb-4e29-82b2-c124947ce205 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:18.041095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f228070f-aecb-4e29-82b2-c124947ce205 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:18.052327 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1253/2499] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:59:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:59:18.056839 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7bce9-0821-4a9b-b8c5-423518a27ec1 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:18.066286 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7bce9-0821-4a9b-b8c5-423518a27ec1 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:18.071771 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7fb2d-8a99-4283-9f5a-3d66c46983e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:18.072368 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7fb2d-8a99-4283-9f5a-3d66c46983e3 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:18.072652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7fb2d-8a99-4283-9f5a-3d66c46983e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:18.072929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7fb2d-8a99-4283-9f5a-3d66c46983e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:18.073589 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e7fb2d-8a99-4283-9f5a-3d66c46983e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:18.075503 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e7fb2d-8a99-4283-9f5a-3d66c46983e3 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:18.105131 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1247/2500] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:59:18 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:59:18.108441 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f228070f-aecb-4e29-82b2-c124947ce205 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:18.118955 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f228070f-aecb-4e29-82b2-c124947ce205 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:18.174818 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b8ff69-e0da-424c-98e5-f4b38d8b30e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:18.175488 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8ff69-e0da-424c-98e5-f4b38d8b30e5 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:18.175617 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8ff69-e0da-424c-98e5-f4b38d8b30e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:18.175888 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8ff69-e0da-424c-98e5-f4b38d8b30e5 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:18.176513 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b8ff69-e0da-424c-98e5-f4b38d8b30e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:18.178509 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b8ff69-e0da-424c-98e5-f4b38d8b30e5 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:18.199403 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1254/2501] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:59:18 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:59:18.203567 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7fb2d-8a99-4283-9f5a-3d66c46983e3 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:18.214470 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7fb2d-8a99-4283-9f5a-3d66c46983e3 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:18.219001 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf84648-8e51-4e9a-8f26-d9cd435891f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:18.219591 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf84648-8e51-4e9a-8f26-d9cd435891f3 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:18.219866 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf84648-8e51-4e9a-8f26-d9cd435891f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:18.220136 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf84648-8e51-4e9a-8f26-d9cd435891f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:18.220763 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf84648-8e51-4e9a-8f26-d9cd435891f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:18.222377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf84648-8e51-4e9a-8f26-d9cd435891f3 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:18.232344 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1248/2502] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:59:18.236377 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b8ff69-e0da-424c-98e5-f4b38d8b30e5 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:18.246665 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b8ff69-e0da-424c-98e5-f4b38d8b30e5 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:18.251701 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6488452c-4d01-4837-8420-3a296ed7a0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:18.252323 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6488452c-4d01-4837-8420-3a296ed7a0b3 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:18.252600 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6488452c-4d01-4837-8420-3a296ed7a0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:18.252868 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6488452c-4d01-4837-8420-3a296ed7a0b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:18.253622 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6488452c-4d01-4837-8420-3a296ed7a0b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:18.255505 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6488452c-4d01-4837-8420-3a296ed7a0b3 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:18.266675 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1255/2503] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:59:18.270942 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf84648-8e51-4e9a-8f26-d9cd435891f3 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:18.281202 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf84648-8e51-4e9a-8f26-d9cd435891f3 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:18.285965 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb283584-7d75-4e02-8c10-e73efab7d821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:18.286602 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb283584-7d75-4e02-8c10-e73efab7d821 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:18.286880 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb283584-7d75-4e02-8c10-e73efab7d821 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:18.287146 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb283584-7d75-4e02-8c10-e73efab7d821 admin admin] PATH_INFO: `/v3/projects/f53e8b325db043f595f6111cb50d09d2/users/62be180368404d2c83f7543264338d4c/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:18.301201 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb283584-7d75-4e02-8c10-e73efab7d821 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f53e8b325db043f595f6111cb50d09d2, user_id=62be180368404d2c83f7543264338d4c, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:18.303865 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb283584-7d75-4e02-8c10-e73efab7d821 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:18.317693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1249/2504] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:59:18 2022] PUT /identity/v3/projects/f53e8b325db043f595f6111cb50d09d2/users/62be180368404d2c83f7543264338d4c/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:59:18.329343 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32600a5-c8f2-41f4-acd6-0e0f9fc2ae1d None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:18.329627 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32600a5-c8f2-41f4-acd6-0e0f9fc2ae1d None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:18.329901 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32600a5-c8f2-41f4-acd6-0e0f9fc2ae1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:18.357754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-b32600a5-c8f2-41f4-acd6-0e0f9fc2ae1d None None] MFA Rules not processed for user `62be180368404d2c83f7543264338d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:59:18.358972 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-b32600a5-c8f2-41f4-acd6-0e0f9fc2ae1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:18.408136 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:18.426353 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1256/2505] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:59:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:59:18.431637 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb283584-7d75-4e02-8c10-e73efab7d821 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:18.493183 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb283584-7d75-4e02-8c10-e73efab7d821 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:18.497918 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7581604-9dfd-44ec-8959-d88716ea97a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:18.498516 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7581604-9dfd-44ec-8959-d88716ea97a0 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:18.498797 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7581604-9dfd-44ec-8959-d88716ea97a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:18.499069 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7581604-9dfd-44ec-8959-d88716ea97a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:18.499950 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7581604-9dfd-44ec-8959-d88716ea97a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:18.501889 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7581604-9dfd-44ec-8959-d88716ea97a0 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:18.528593 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1250/2506] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:59:18 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:59:18.532482 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32600a5-c8f2-41f4-acd6-0e0f9fc2ae1d None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:18.543198 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32600a5-c8f2-41f4-acd6-0e0f9fc2ae1d None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:18.600620 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf7392-d877-45c1-a0ae-25ada6679c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:18.601225 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf7392-d877-45c1-a0ae-25ada6679c21 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:18.601562 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf7392-d877-45c1-a0ae-25ada6679c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:18.601843 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf7392-d877-45c1-a0ae-25ada6679c21 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:18.602476 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf7392-d877-45c1-a0ae-25ada6679c21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:18.604384 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf7392-d877-45c1-a0ae-25ada6679c21 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:18.625906 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1257/2507] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:59:18 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:59:18.630004 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7581604-9dfd-44ec-8959-d88716ea97a0 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:18.640278 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7581604-9dfd-44ec-8959-d88716ea97a0 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:18.644817 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907009bb-ee5b-4d33-8b78-915280b03728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:18.645448 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907009bb-ee5b-4d33-8b78-915280b03728 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:18.645725 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907009bb-ee5b-4d33-8b78-915280b03728 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:18.645991 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907009bb-ee5b-4d33-8b78-915280b03728 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:18.646617 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907009bb-ee5b-4d33-8b78-915280b03728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:18.648971 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907009bb-ee5b-4d33-8b78-915280b03728 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:18.659834 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1251/2508] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:59:18.663913 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf7392-d877-45c1-a0ae-25ada6679c21 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:18.674340 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf7392-d877-45c1-a0ae-25ada6679c21 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:18.679062 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26dd500-0d62-4a1e-a632-c557d24bee93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:18.679748 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26dd500-0d62-4a1e-a632-c557d24bee93 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:18.680047 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26dd500-0d62-4a1e-a632-c557d24bee93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:18.680322 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26dd500-0d62-4a1e-a632-c557d24bee93 admin admin] PATH_INFO: `/v3/projects/764f9b807b414fe498386541d2215691/users/4bb301bc24394f4ebb5f2b1c2490f1b1/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:18.694762 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26dd500-0d62-4a1e-a632-c557d24bee93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=764f9b807b414fe498386541d2215691, user_id=4bb301bc24394f4ebb5f2b1c2490f1b1, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:18.698005 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26dd500-0d62-4a1e-a632-c557d24bee93 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:18.712328 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1258/2509] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:59:18 2022] PUT /identity/v3/projects/764f9b807b414fe498386541d2215691/users/4bb301bc24394f4ebb5f2b1c2490f1b1/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:18.717055 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907009bb-ee5b-4d33-8b78-915280b03728 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:18.728547 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907009bb-ee5b-4d33-8b78-915280b03728 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:18.760888 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:18.787303 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b99b126-881b-4707-8535-8ba184775713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:18.787907 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99b126-881b-4707-8535-8ba184775713 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:18.788183 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99b126-881b-4707-8535-8ba184775713 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:18.788454 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99b126-881b-4707-8535-8ba184775713 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:18.789084 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b99b126-881b-4707-8535-8ba184775713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:18.790712 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b99b126-881b-4707-8535-8ba184775713 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:18.800386 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1252/2510] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 22:59:18.804614 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26dd500-0d62-4a1e-a632-c557d24bee93 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:18.814881 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26dd500-0d62-4a1e-a632-c557d24bee93 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:18.819504 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77789268-5216-452d-a67a-0f1147a04012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:18.820153 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77789268-5216-452d-a67a-0f1147a04012 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:18.821006 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77789268-5216-452d-a67a-0f1147a04012 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:18.821303 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77789268-5216-452d-a67a-0f1147a04012 admin admin] PATH_INFO: `/v3/projects/764f9b807b414fe498386541d2215691/users/4bb301bc24394f4ebb5f2b1c2490f1b1/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:18.823297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77789268-5216-452d-a67a-0f1147a04012 admin admin] RBAC: Authorizing `identity:create_grant(project_id=764f9b807b414fe498386541d2215691, user_id=4bb301bc24394f4ebb5f2b1c2490f1b1, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:18.825987 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77789268-5216-452d-a67a-0f1147a04012 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:18.836650 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1259/2511] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:59:18 2022] PUT /identity/v3/projects/764f9b807b414fe498386541d2215691/users/4bb301bc24394f4ebb5f2b1c2490f1b1/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:18.848209 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7bc8d-a778-475b-98c8-baf82ac61b3e None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:18.848497 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7bc8d-a778-475b-98c8-baf82ac61b3e None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:18.848773 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7bc8d-a778-475b-98c8-baf82ac61b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:18.876421 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-9cb7bc8d-a778-475b-98c8-baf82ac61b3e None None] MFA Rules not processed for user `4bb301bc24394f4ebb5f2b1c2490f1b1`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:59:18.877671 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-9cb7bc8d-a778-475b-98c8-baf82ac61b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:18.944509 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1253/2512] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:59:18 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:59:18.957897 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77789268-5216-452d-a67a-0f1147a04012 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:18.959611 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-77789268-5216-452d-a67a-0f1147a04012 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:19.016815 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77789268-5216-452d-a67a-0f1147a04012 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:19.020478 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3910b7f-4fbd-43d7-a98d-6811f98065aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:19.021073 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3910b7f-4fbd-43d7-a98d-6811f98065aa service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:19.021381 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3910b7f-4fbd-43d7-a98d-6811f98065aa service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:19.021655 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3910b7f-4fbd-43d7-a98d-6811f98065aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:19.031252 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3910b7f-4fbd-43d7-a98d-6811f98065aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:19.032671 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3910b7f-4fbd-43d7-a98d-6811f98065aa service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:19.042352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1260/2513] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:59:18 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:59:19.347792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb7bc8d-a778-475b-98c8-baf82ac61b3e None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:19.357731 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb7bc8d-a778-475b-98c8-baf82ac61b3e None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:19.362345 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39b0d06-235d-4b85-a29a-baf92177e5a8 tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bb301bc24394f4ebb5f2b1c2490f1b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '764f9b807b414fe498386541d2215691', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:19.363065 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b0d06-235d-4b85-a29a-baf92177e5a8 tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:19.363385 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b0d06-235d-4b85-a29a-baf92177e5a8 tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:19.363657 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b0d06-235d-4b85-a29a-baf92177e5a8 tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:19.364291 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39b0d06-235d-4b85-a29a-baf92177e5a8 tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:59:19.364758 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39b0d06-235d-4b85-a29a-baf92177e5a8 tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:19.366390 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39b0d06-235d-4b85-a29a-baf92177e5a8 tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:19.378361 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1254/2514] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:59:19.382557 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3910b7f-4fbd-43d7-a98d-6811f98065aa service cinder] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:19.391712 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3910b7f-4fbd-43d7-a98d-6811f98065aa service cinder] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:19.396297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ede774-3200-4079-b6bc-82fea526a3d1 tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bb301bc24394f4ebb5f2b1c2490f1b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '764f9b807b414fe498386541d2215691', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:19.396884 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ede774-3200-4079-b6bc-82fea526a3d1 tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:19.397168 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ede774-3200-4079-b6bc-82fea526a3d1 tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:19.397473 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ede774-3200-4079-b6bc-82fea526a3d1 tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:19.398085 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ede774-3200-4079-b6bc-82fea526a3d1 tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:19.400619 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ede774-3200-4079-b6bc-82fea526a3d1 tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:19.428190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1261/2515] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:59:19 2022] POST /identity/v3/projects => generated 354 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:59:19.433080 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39b0d06-235d-4b85-a29a-baf92177e5a8 tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:19.443571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39b0d06-235d-4b85-a29a-baf92177e5a8 tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:19.501670 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203e568-3067-499c-b67b-7d295961d286 tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bb301bc24394f4ebb5f2b1c2490f1b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '764f9b807b414fe498386541d2215691', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:19.502201 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203e568-3067-499c-b67b-7d295961d286 tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:19.502487 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203e568-3067-499c-b67b-7d295961d286 tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:19.502767 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203e568-3067-499c-b67b-7d295961d286 tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:19.503413 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7203e568-3067-499c-b67b-7d295961d286 tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:59:19.503722 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7203e568-3067-499c-b67b-7d295961d286 tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:19.505364 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7203e568-3067-499c-b67b-7d295961d286 tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:19.517631 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1255/2516] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:59:19.553912 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ede774-3200-4079-b6bc-82fea526a3d1 tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:19.565627 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ede774-3200-4079-b6bc-82fea526a3d1 tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:19.570264 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e1eec0-83ed-4d76-bfb1-edcc6044b12f tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bb301bc24394f4ebb5f2b1c2490f1b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '764f9b807b414fe498386541d2215691', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:19.571028 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1eec0-83ed-4d76-bfb1-edcc6044b12f tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:19.571309 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1eec0-83ed-4d76-bfb1-edcc6044b12f tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:19.571582 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1eec0-83ed-4d76-bfb1-edcc6044b12f tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] PATH_INFO: `/v3/projects/8cc55b69815e4677843df06f1ddec544` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:19.572714 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e1eec0-83ed-4d76-bfb1-edcc6044b12f tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8cc55b69815e4677843df06f1ddec544)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:19.574654 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e1eec0-83ed-4d76-bfb1-edcc6044b12f tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:19.595075 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:19.621389 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-05e1eec0-83ed-4d76-bfb1-edcc6044b12f tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cc55b69815e4677843df06f1ddec544', 'request_id': 'req-05e1eec0-83ed-4d76-bfb1-edcc6044b12f'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:19.627308 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-05e1eec0-83ed-4d76-bfb1-edcc6044b12f tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cc55b69815e4677843df06f1ddec544', 'request_id': 'req-05e1eec0-83ed-4d76-bfb1-edcc6044b12f'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:19.635530 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-05e1eec0-83ed-4d76-bfb1-edcc6044b12f tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cc55b69815e4677843df06f1ddec544', 'request_id': 'req-05e1eec0-83ed-4d76-bfb1-edcc6044b12f'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:19.647091 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-05e1eec0-83ed-4d76-bfb1-edcc6044b12f tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] The token cache is being invalidate because project 8cc55b69815e4677843df06f1ddec544 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:59:19.647404 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-05e1eec0-83ed-4d76-bfb1-edcc6044b12f tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:19.648056 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-05e1eec0-83ed-4d76-bfb1-edcc6044b12f tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:19.650205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1262/2517] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:59:19 2022] DELETE /identity/v3/projects/8cc55b69815e4677843df06f1ddec544 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:19.843385 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203e568-3067-499c-b67b-7d295961d286 tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:19.845155 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-7203e568-3067-499c-b67b-7d295961d286 tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:19.888805 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:19.912897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203e568-3067-499c-b67b-7d295961d286 tempest-VolumeQuotaClassesTest-1339401470 tempest-VolumeQuotaClassesTest-1339401470-project-admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:19.967920 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fcfc73-c4c4-4a32-8c75-5dcc141058f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:19.968538 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fcfc73-c4c4-4a32-8c75-5dcc141058f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:19.968815 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fcfc73-c4c4-4a32-8c75-5dcc141058f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:19.969081 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fcfc73-c4c4-4a32-8c75-5dcc141058f4 admin admin] PATH_INFO: `/v3/users/62be180368404d2c83f7543264338d4c` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:19.970218 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fcfc73-c4c4-4a32-8c75-5dcc141058f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62be180368404d2c83f7543264338d4c)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:19.972123 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fcfc73-c4c4-4a32-8c75-5dcc141058f4 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:20.016195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-27fcfc73-c4c4-4a32-8c75-5dcc141058f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62be180368404d2c83f7543264338d4c'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:20.022843 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-27fcfc73-c4c4-4a32-8c75-5dcc141058f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62be180368404d2c83f7543264338d4c'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:20.035297 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-27fcfc73-c4c4-4a32-8c75-5dcc141058f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62be180368404d2c83f7543264338d4c'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:20.035980 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-27fcfc73-c4c4-4a32-8c75-5dcc141058f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62be180368404d2c83f7543264338d4c'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:20.036583 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-27fcfc73-c4c4-4a32-8c75-5dcc141058f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62be180368404d2c83f7543264338d4c'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:20.050086 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1256/2518] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:59:19 2022] DELETE /identity/v3/users/62be180368404d2c83f7543264338d4c => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:20.054798 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09aded9-6e63-4542-96ef-92adc5efa0e1 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:20.055081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09aded9-6e63-4542-96ef-92adc5efa0e1 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:20.055460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09aded9-6e63-4542-96ef-92adc5efa0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:20.074500 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-b09aded9-6e63-4542-96ef-92adc5efa0e1 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:59:20.075715 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-b09aded9-6e63-4542-96ef-92adc5efa0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:20.138646 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1263/2519] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:59:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:59:20.148838 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fcfc73-c4c4-4a32-8c75-5dcc141058f4 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:20.150446 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-27fcfc73-c4c4-4a32-8c75-5dcc141058f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:20.214945 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fcfc73-c4c4-4a32-8c75-5dcc141058f4 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:20.218572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94af5c1-57cc-4913-85f9-82d8d135c5ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:20.219170 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94af5c1-57cc-4913-85f9-82d8d135c5ce service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:20.219446 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94af5c1-57cc-4913-85f9-82d8d135c5ce service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:20.219715 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94af5c1-57cc-4913-85f9-82d8d135c5ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:20.228689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94af5c1-57cc-4913-85f9-82d8d135c5ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:20.230142 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94af5c1-57cc-4913-85f9-82d8d135c5ce service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:20.239399 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1257/2520] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:59:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:59:20.509799 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09aded9-6e63-4542-96ef-92adc5efa0e1 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:20.511355 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-b09aded9-6e63-4542-96ef-92adc5efa0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:20.524164 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09aded9-6e63-4542-96ef-92adc5efa0e1 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:20.529113 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2cc09-2cf1-43e1-894f-261643a727c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:20.529810 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2cc09-2cf1-43e1-894f-261643a727c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:20.530089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2cc09-2cf1-43e1-894f-261643a727c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:20.530359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2cc09-2cf1-43e1-894f-261643a727c7 admin admin] PATH_INFO: `/v3/projects/f53e8b325db043f595f6111cb50d09d2` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:20.531482 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2cc09-2cf1-43e1-894f-261643a727c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f53e8b325db043f595f6111cb50d09d2)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:20.533436 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2cc09-2cf1-43e1-894f-261643a727c7 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:20.580607 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-97e2cc09-2cf1-43e1-894f-261643a727c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f53e8b325db043f595f6111cb50d09d2', 'request_id': 'req-97e2cc09-2cf1-43e1-894f-261643a727c7'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:20.586116 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-97e2cc09-2cf1-43e1-894f-261643a727c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f53e8b325db043f595f6111cb50d09d2', 'request_id': 'req-97e2cc09-2cf1-43e1-894f-261643a727c7'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:20.593198 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-97e2cc09-2cf1-43e1-894f-261643a727c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f53e8b325db043f595f6111cb50d09d2', 'request_id': 'req-97e2cc09-2cf1-43e1-894f-261643a727c7'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:20.604885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-97e2cc09-2cf1-43e1-894f-261643a727c7 admin admin] The token cache is being invalidate because project f53e8b325db043f595f6111cb50d09d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:59:20.605174 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-97e2cc09-2cf1-43e1-894f-261643a727c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:20.605872 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-97e2cc09-2cf1-43e1-894f-261643a727c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:20.607366 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1264/2521] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:59:20 2022] DELETE /identity/v3/projects/f53e8b325db043f595f6111cb50d09d2 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:20.611549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94af5c1-57cc-4913-85f9-82d8d135c5ce service neutron] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:20.622137 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94af5c1-57cc-4913-85f9-82d8d135c5ce service neutron] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:20.626160 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:20.681628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01962983-ba24-41f9-85e7-62d820f60b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:20.682259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01962983-ba24-41f9-85e7-62d820f60b1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:20.682722 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01962983-ba24-41f9-85e7-62d820f60b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:20.682855 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01962983-ba24-41f9-85e7-62d820f60b1d admin admin] PATH_INFO: `/v3/users/4bb301bc24394f4ebb5f2b1c2490f1b1` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:20.683969 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01962983-ba24-41f9-85e7-62d820f60b1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bb301bc24394f4ebb5f2b1c2490f1b1)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:20.686000 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01962983-ba24-41f9-85e7-62d820f60b1d admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:20.730997 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-01962983-ba24-41f9-85e7-62d820f60b1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bb301bc24394f4ebb5f2b1c2490f1b1'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:20.737802 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-01962983-ba24-41f9-85e7-62d820f60b1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bb301bc24394f4ebb5f2b1c2490f1b1'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:20.750538 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-01962983-ba24-41f9-85e7-62d820f60b1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bb301bc24394f4ebb5f2b1c2490f1b1'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:20.751276 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-01962983-ba24-41f9-85e7-62d820f60b1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bb301bc24394f4ebb5f2b1c2490f1b1'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:20.751888 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-01962983-ba24-41f9-85e7-62d820f60b1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bb301bc24394f4ebb5f2b1c2490f1b1'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:20.764686 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1258/2522] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:59:20 2022] DELETE /identity/v3/users/4bb301bc24394f4ebb5f2b1c2490f1b1 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:21.047477 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2cc09-2cf1-43e1-894f-261643a727c7 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:21.049319 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-97e2cc09-2cf1-43e1-894f-261643a727c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:21.062787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2cc09-2cf1-43e1-894f-261643a727c7 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:21.094119 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:21.123285 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aac95f-7aea-4d32-86de-6c16bc26d3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:21.123844 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aac95f-7aea-4d32-86de-6c16bc26d3b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:21.124389 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aac95f-7aea-4d32-86de-6c16bc26d3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:21.124662 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aac95f-7aea-4d32-86de-6c16bc26d3b6 admin admin] PATH_INFO: `/v3/projects/764f9b807b414fe498386541d2215691` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:21.125808 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aac95f-7aea-4d32-86de-6c16bc26d3b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=764f9b807b414fe498386541d2215691)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:21.127717 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aac95f-7aea-4d32-86de-6c16bc26d3b6 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:21.174881 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-45aac95f-7aea-4d32-86de-6c16bc26d3b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '764f9b807b414fe498386541d2215691', 'request_id': 'req-45aac95f-7aea-4d32-86de-6c16bc26d3b6'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:21.180437 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-45aac95f-7aea-4d32-86de-6c16bc26d3b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '764f9b807b414fe498386541d2215691', 'request_id': 'req-45aac95f-7aea-4d32-86de-6c16bc26d3b6'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:21.187705 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-45aac95f-7aea-4d32-86de-6c16bc26d3b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '764f9b807b414fe498386541d2215691', 'request_id': 'req-45aac95f-7aea-4d32-86de-6c16bc26d3b6'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:21.199665 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-45aac95f-7aea-4d32-86de-6c16bc26d3b6 admin admin] The token cache is being invalidate because project 764f9b807b414fe498386541d2215691 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:59:21.200123 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-45aac95f-7aea-4d32-86de-6c16bc26d3b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:21.200658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-45aac95f-7aea-4d32-86de-6c16bc26d3b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:21.202310 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1265/2523] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:59:21 2022] DELETE /identity/v3/projects/764f9b807b414fe498386541d2215691 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:59:21.207262 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8fb739-82a8-454a-ac6f-37bda57ff58b None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:21.207551 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8fb739-82a8-454a-ac6f-37bda57ff58b None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:21.207826 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8fb739-82a8-454a-ac6f-37bda57ff58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:21.227903 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-0f8fb739-82a8-454a-ac6f-37bda57ff58b None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:59:21.229110 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-0f8fb739-82a8-454a-ac6f-37bda57ff58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:21.291281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1259/2524] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:59:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:59:21.299135 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823f347-ae78-458a-bf5a-7f83523122a2 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:21.299418 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823f347-ae78-458a-bf5a-7f83523122a2 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:21.299692 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823f347-ae78-458a-bf5a-7f83523122a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:21.319970 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-3823f347-ae78-458a-bf5a-7f83523122a2 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:59:21.321164 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-3823f347-ae78-458a-bf5a-7f83523122a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:21.329409 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1266/2525] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:59:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:59:21.333593 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8fb739-82a8-454a-ac6f-37bda57ff58b None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:21.343859 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8fb739-82a8-454a-ac6f-37bda57ff58b None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:21.348535 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da1504-59f7-428b-96c9-9a7a203959ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:21.349126 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da1504-59f7-428b-96c9-9a7a203959ef admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:21.349441 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da1504-59f7-428b-96c9-9a7a203959ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:21.349727 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da1504-59f7-428b-96c9-9a7a203959ef admin admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:21.350365 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da1504-59f7-428b-96c9-9a7a203959ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:59:21.350664 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da1504-59f7-428b-96c9-9a7a203959ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:21.352910 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da1504-59f7-428b-96c9-9a7a203959ef admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:21.364921 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1260/2526] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:59:21.369720 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3823f347-ae78-458a-bf5a-7f83523122a2 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:21.379352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3823f347-ae78-458a-bf5a-7f83523122a2 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:21.384223 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e919cc-ba4e-4a1e-a9fb-975d12e32e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:21.384827 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e919cc-ba4e-4a1e-a9fb-975d12e32e6b admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:21.385110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e919cc-ba4e-4a1e-a9fb-975d12e32e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:21.385428 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e919cc-ba4e-4a1e-a9fb-975d12e32e6b admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:21.386064 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e919cc-ba4e-4a1e-a9fb-975d12e32e6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:21.387971 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e919cc-ba4e-4a1e-a9fb-975d12e32e6b admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:21.419791 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1267/2527] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:59:21 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:59:21.422561 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da1504-59f7-428b-96c9-9a7a203959ef admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:21.432935 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da1504-59f7-428b-96c9-9a7a203959ef admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:21.442065 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:21.498528 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8244398-d123-40dc-a9da-aaabbc7334cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:21.499688 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8244398-d123-40dc-a9da-aaabbc7334cf admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:21.500117 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8244398-d123-40dc-a9da-aaabbc7334cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:21.500533 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8244398-d123-40dc-a9da-aaabbc7334cf admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:21.501522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8244398-d123-40dc-a9da-aaabbc7334cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:21.504610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8244398-d123-40dc-a9da-aaabbc7334cf admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:21.531242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1261/2528] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:59:21 2022] POST /identity/v3/users => generated 294 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:59:21.534841 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e919cc-ba4e-4a1e-a9fb-975d12e32e6b admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:21.546154 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e919cc-ba4e-4a1e-a9fb-975d12e32e6b admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:21.550930 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a6752-f1b3-4707-8b02-56ded1a44f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:21.551552 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a6752-f1b3-4707-8b02-56ded1a44f15 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:21.551835 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a6752-f1b3-4707-8b02-56ded1a44f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:21.552113 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a6752-f1b3-4707-8b02-56ded1a44f15 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:21.552882 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5a6752-f1b3-4707-8b02-56ded1a44f15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:21.554791 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5a6752-f1b3-4707-8b02-56ded1a44f15 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:21.565948 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1268/2529] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:59:21.570045 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8244398-d123-40dc-a9da-aaabbc7334cf admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:21.580497 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8244398-d123-40dc-a9da-aaabbc7334cf admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:21.585242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6305a67-ef49-4583-8bb7-0ee0d049d83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:21.586598 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6305a67-ef49-4583-8bb7-0ee0d049d83e admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:21.586966 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6305a67-ef49-4583-8bb7-0ee0d049d83e admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:21.587245 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6305a67-ef49-4583-8bb7-0ee0d049d83e admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:21.587888 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6305a67-ef49-4583-8bb7-0ee0d049d83e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:21.589526 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6305a67-ef49-4583-8bb7-0ee0d049d83e admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:21.600154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1262/2530] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:59:21.604946 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a6752-f1b3-4707-8b02-56ded1a44f15 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:21.614892 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a6752-f1b3-4707-8b02-56ded1a44f15 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:21.620029 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4bedc5-e77d-4047-8cab-9f85e862d2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:21.620739 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4bedc5-e77d-4047-8cab-9f85e862d2eb admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:21.621026 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4bedc5-e77d-4047-8cab-9f85e862d2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:21.621327 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4bedc5-e77d-4047-8cab-9f85e862d2eb admin admin] PATH_INFO: `/v3/projects/5355372a696a4957b8c7efcd657e2c29/users/99be145c05a74cf081dceba537d46bf9/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:21.636704 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4bedc5-e77d-4047-8cab-9f85e862d2eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5355372a696a4957b8c7efcd657e2c29, user_id=99be145c05a74cf081dceba537d46bf9, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:21.639381 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4bedc5-e77d-4047-8cab-9f85e862d2eb admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:21.652806 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1269/2531] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:59:21 2022] PUT /identity/v3/projects/5355372a696a4957b8c7efcd657e2c29/users/99be145c05a74cf081dceba537d46bf9/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:21.665090 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3aca6-d609-45d6-9ed4-52099a57081a None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:21.665671 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3aca6-d609-45d6-9ed4-52099a57081a None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:21.665671 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3aca6-d609-45d6-9ed4-52099a57081a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:21.692595 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-c9b3aca6-d609-45d6-9ed4-52099a57081a None None] MFA Rules not processed for user `99be145c05a74cf081dceba537d46bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:59:21.693836 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-c9b3aca6-d609-45d6-9ed4-52099a57081a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:21.759396 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1263/2532] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:59:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:59:21.763892 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4bedc5-e77d-4047-8cab-9f85e862d2eb admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:21.822738 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:21.827167 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4bedc5-e77d-4047-8cab-9f85e862d2eb admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:21.833451 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60a2b9-ef13-4351-b96d-e5a34142865f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:21.834440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60a2b9-ef13-4351-b96d-e5a34142865f admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:21.834781 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60a2b9-ef13-4351-b96d-e5a34142865f admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:21.835255 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60a2b9-ef13-4351-b96d-e5a34142865f admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:21.838367 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e60a2b9-ef13-4351-b96d-e5a34142865f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:21.842069 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e60a2b9-ef13-4351-b96d-e5a34142865f admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:21.871368 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1270/2533] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:59:21 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:59:21.875716 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3aca6-d609-45d6-9ed4-52099a57081a None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:21.887255 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3aca6-d609-45d6-9ed4-52099a57081a None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:21.945011 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca3562-d578-44d7-877f-63bfb05423a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:21.945643 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca3562-d578-44d7-877f-63bfb05423a1 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:21.945922 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca3562-d578-44d7-877f-63bfb05423a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:21.946195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca3562-d578-44d7-877f-63bfb05423a1 admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:21.946811 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca3562-d578-44d7-877f-63bfb05423a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:21.948742 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca3562-d578-44d7-877f-63bfb05423a1 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:21.968873 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1264/2534] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:59:21 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:59:21.972892 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60a2b9-ef13-4351-b96d-e5a34142865f admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:21.983711 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60a2b9-ef13-4351-b96d-e5a34142865f admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:21.988285 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9670651-6bea-4360-877d-0d1642f202eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:21.988876 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9670651-6bea-4360-877d-0d1642f202eb admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:21.989152 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9670651-6bea-4360-877d-0d1642f202eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:21.989462 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9670651-6bea-4360-877d-0d1642f202eb admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:21.990091 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9670651-6bea-4360-877d-0d1642f202eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:21.991676 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9670651-6bea-4360-877d-0d1642f202eb admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:22.002873 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1271/2535] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:59:22.006936 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca3562-d578-44d7-877f-63bfb05423a1 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:22.016872 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca3562-d578-44d7-877f-63bfb05423a1 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:22.021463 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd0f10-f815-4841-b6b9-064987c11ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:22.022053 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd0f10-f815-4841-b6b9-064987c11ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:22.022332 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd0f10-f815-4841-b6b9-064987c11ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:22.022603 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd0f10-f815-4841-b6b9-064987c11ea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:22.023283 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd0f10-f815-4841-b6b9-064987c11ea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:22.025088 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd0f10-f815-4841-b6b9-064987c11ea7 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:22.030233 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:22.035881 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1265/2536] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:59:22.040505 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9670651-6bea-4360-877d-0d1642f202eb admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:22.050352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9670651-6bea-4360-877d-0d1642f202eb admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:22.055591 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e94b81-aca2-4fa6-b088-660a963b724a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:22.056253 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e94b81-aca2-4fa6-b088-660a963b724a admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:22.056510 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e94b81-aca2-4fa6-b088-660a963b724a admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:22.056778 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e94b81-aca2-4fa6-b088-660a963b724a admin admin] PATH_INFO: `/v3/projects/b267403c20944393b1b71ce0a731954d/users/73a8f5c5b7624b3f905e1241fa6efd4c/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:22.070968 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e94b81-aca2-4fa6-b088-660a963b724a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b267403c20944393b1b71ce0a731954d, user_id=73a8f5c5b7624b3f905e1241fa6efd4c, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:22.073676 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e94b81-aca2-4fa6-b088-660a963b724a admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:22.087745 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1272/2537] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:59:22 2022] PUT /identity/v3/projects/b267403c20944393b1b71ce0a731954d/users/73a8f5c5b7624b3f905e1241fa6efd4c/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:59:22.098423 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fda1e-05fb-4252-91d7-b5cae8f1e46e None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:22.098710 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fda1e-05fb-4252-91d7-b5cae8f1e46e None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:22.098986 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fda1e-05fb-4252-91d7-b5cae8f1e46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:22.127367 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-053fda1e-05fb-4252-91d7-b5cae8f1e46e None None] MFA Rules not processed for user `73a8f5c5b7624b3f905e1241fa6efd4c`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:59:22.128581 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-053fda1e-05fb-4252-91d7-b5cae8f1e46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:22.193914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1266/2538] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:59:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:59:22.199254 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e94b81-aca2-4fa6-b088-660a963b724a admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:22.261706 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e94b81-aca2-4fa6-b088-660a963b724a admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:22.266386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed324b-8083-4b7d-a248-6a67c8892711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:22.266989 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed324b-8083-4b7d-a248-6a67c8892711 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:22.267283 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed324b-8083-4b7d-a248-6a67c8892711 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:22.267556 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed324b-8083-4b7d-a248-6a67c8892711 admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:22.268182 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bed324b-8083-4b7d-a248-6a67c8892711 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:22.270139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bed324b-8083-4b7d-a248-6a67c8892711 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:22.296001 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1273/2539] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:59:22 2022] POST /identity/v3/projects => generated 387 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:59:22.300191 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053fda1e-05fb-4252-91d7-b5cae8f1e46e None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:22.311567 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053fda1e-05fb-4252-91d7-b5cae8f1e46e None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:22.365845 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a6532-e612-4372-be19-77fc615d4e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:22.366594 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a6532-e612-4372-be19-77fc615d4e0b admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:22.366908 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a6532-e612-4372-be19-77fc615d4e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:22.367183 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a6532-e612-4372-be19-77fc615d4e0b admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:22.367820 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a6532-e612-4372-be19-77fc615d4e0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:22.369936 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a6532-e612-4372-be19-77fc615d4e0b admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:22.391011 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1267/2540] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:59:22 2022] POST /identity/v3/users => generated 300 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:59:22.394995 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed324b-8083-4b7d-a248-6a67c8892711 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:22.405926 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed324b-8083-4b7d-a248-6a67c8892711 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:22.410514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd0436-b1c3-41b0-ad13-2327008fe77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:22.411105 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd0436-b1c3-41b0-ad13-2327008fe77a admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:22.411384 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd0436-b1c3-41b0-ad13-2327008fe77a admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:22.411655 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd0436-b1c3-41b0-ad13-2327008fe77a admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:22.412286 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd0436-b1c3-41b0-ad13-2327008fe77a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:22.414550 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd0436-b1c3-41b0-ad13-2327008fe77a admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:22.425906 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1274/2541] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:59:22.430658 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a6532-e612-4372-be19-77fc615d4e0b admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:22.440958 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a6532-e612-4372-be19-77fc615d4e0b admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:22.446526 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7443e4-b643-40db-8814-cde33f4dfe2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:22.447121 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7443e4-b643-40db-8814-cde33f4dfe2a admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:22.447405 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7443e4-b643-40db-8814-cde33f4dfe2a admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:22.447683 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7443e4-b643-40db-8814-cde33f4dfe2a admin admin] PATH_INFO: `/v3/projects/af0a3b61594b4065b03a319dbb42625a/users/33ac85866a6849db8c1766bafe6193bf/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:22.463610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7443e4-b643-40db-8814-cde33f4dfe2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=af0a3b61594b4065b03a319dbb42625a, user_id=33ac85866a6849db8c1766bafe6193bf, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:22.466341 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7443e4-b643-40db-8814-cde33f4dfe2a admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:22.478826 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1268/2542] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:59:22 2022] PUT /identity/v3/projects/af0a3b61594b4065b03a319dbb42625a/users/33ac85866a6849db8c1766bafe6193bf/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:22.482989 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd0436-b1c3-41b0-ad13-2327008fe77a admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:22.493052 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd0436-b1c3-41b0-ad13-2327008fe77a admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:22.548184 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00304f9-375e-4fd5-984d-b7a302580c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:22.548853 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00304f9-375e-4fd5-984d-b7a302580c2d admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:22.549093 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00304f9-375e-4fd5-984d-b7a302580c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:22.549422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00304f9-375e-4fd5-984d-b7a302580c2d admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:22.550060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00304f9-375e-4fd5-984d-b7a302580c2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:22.551667 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00304f9-375e-4fd5-984d-b7a302580c2d admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:22.561329 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1275/2543] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:59:22.565587 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7443e4-b643-40db-8814-cde33f4dfe2a admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:22.577435 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7443e4-b643-40db-8814-cde33f4dfe2a admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:22.582169 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5806f0-6fa2-4f16-a0af-26c4a878f469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:22.582879 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5806f0-6fa2-4f16-a0af-26c4a878f469 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:22.583157 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5806f0-6fa2-4f16-a0af-26c4a878f469 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:22.583432 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5806f0-6fa2-4f16-a0af-26c4a878f469 admin admin] PATH_INFO: `/v3/projects/af0a3b61594b4065b03a319dbb42625a/users/33ac85866a6849db8c1766bafe6193bf/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:22.586002 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5806f0-6fa2-4f16-a0af-26c4a878f469 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af0a3b61594b4065b03a319dbb42625a, user_id=33ac85866a6849db8c1766bafe6193bf, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:22.588660 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5806f0-6fa2-4f16-a0af-26c4a878f469 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:22.598993 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1269/2544] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:59:22 2022] PUT /identity/v3/projects/af0a3b61594b4065b03a319dbb42625a/users/33ac85866a6849db8c1766bafe6193bf/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:59:22.609957 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74a7c5-ccd1-4ba5-a8bf-0bb7b9238caf None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:22.610245 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74a7c5-ccd1-4ba5-a8bf-0bb7b9238caf None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:22.610521 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74a7c5-ccd1-4ba5-a8bf-0bb7b9238caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:22.638789 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-1a74a7c5-ccd1-4ba5-a8bf-0bb7b9238caf None None] MFA Rules not processed for user `33ac85866a6849db8c1766bafe6193bf`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:59:22.640004 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-1a74a7c5-ccd1-4ba5-a8bf-0bb7b9238caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:22.644971 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:22.708494 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1276/2545] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:59:22 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:59:22.720963 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5806f0-6fa2-4f16-a0af-26c4a878f469 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:22.722758 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-9d5806f0-6fa2-4f16-a0af-26c4a878f469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:22.770193 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:22.783711 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5806f0-6fa2-4f16-a0af-26c4a878f469 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:22.787425 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32da06d9-af0d-47c0-bb89-9da7261d8c62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:22.788025 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da06d9-af0d-47c0-bb89-9da7261d8c62 service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:22.788305 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da06d9-af0d-47c0-bb89-9da7261d8c62 service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:22.788582 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da06d9-af0d-47c0-bb89-9da7261d8c62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:22.797686 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32da06d9-af0d-47c0-bb89-9da7261d8c62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:22.799287 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32da06d9-af0d-47c0-bb89-9da7261d8c62 service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:22.809578 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1270/2546] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:59:22 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:59:23.307850 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a74a7c5-ccd1-4ba5-a8bf-0bb7b9238caf None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:23.309245 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-1a74a7c5-ccd1-4ba5-a8bf-0bb7b9238caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:23.321694 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a74a7c5-ccd1-4ba5-a8bf-0bb7b9238caf None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:23.325334 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d016ea-f3a8-4d7e-b983-a7a8ea984417 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:23.325938 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d016ea-f3a8-4d7e-b983-a7a8ea984417 service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:23.326214 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d016ea-f3a8-4d7e-b983-a7a8ea984417 service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:23.326486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d016ea-f3a8-4d7e-b983-a7a8ea984417 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:23.384303 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d016ea-f3a8-4d7e-b983-a7a8ea984417 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:23.385832 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d016ea-f3a8-4d7e-b983-a7a8ea984417 service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:23.394753 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1277/2547] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:59:23 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:59:27.430603 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32da06d9-af0d-47c0-bb89-9da7261d8c62 service cinder] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:27.439398 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32da06d9-af0d-47c0-bb89-9da7261d8c62 service cinder] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:27.443213 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f35e8-5114-411a-a46f-8b12a736c1a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:27.443808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f35e8-5114-411a-a46f-8b12a736c1a9 service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:27.444083 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f35e8-5114-411a-a46f-8b12a736c1a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:27.444378 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f35e8-5114-411a-a46f-8b12a736c1a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:27.500399 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f35e8-5114-411a-a46f-8b12a736c1a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:27.501864 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f35e8-5114-411a-a46f-8b12a736c1a9 service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:27.511878 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1271/2548] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:59:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:59:29.599338 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d016ea-f3a8-4d7e-b983-a7a8ea984417 service cinder] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:29.664501 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d016ea-f3a8-4d7e-b983-a7a8ea984417 service cinder] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:29.669296 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e46ee-df2c-443c-af2e-ea57608d2de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:29.669945 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e46ee-df2c-443c-af2e-ea57608d2de7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:29.670227 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e46ee-df2c-443c-af2e-ea57608d2de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:29.670500 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e46ee-df2c-443c-af2e-ea57608d2de7 admin admin] PATH_INFO: `/v3/users/99be145c05a74cf081dceba537d46bf9` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:29.671619 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e46ee-df2c-443c-af2e-ea57608d2de7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99be145c05a74cf081dceba537d46bf9)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:29.673587 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e46ee-df2c-443c-af2e-ea57608d2de7 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:29.716839 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:29.719432 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-ee7e46ee-df2c-443c-af2e-ea57608d2de7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99be145c05a74cf081dceba537d46bf9'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:29.727604 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-ee7e46ee-df2c-443c-af2e-ea57608d2de7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99be145c05a74cf081dceba537d46bf9'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:29.735596 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-ee7e46ee-df2c-443c-af2e-ea57608d2de7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99be145c05a74cf081dceba537d46bf9'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:29.736264 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-ee7e46ee-df2c-443c-af2e-ea57608d2de7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99be145c05a74cf081dceba537d46bf9'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:29.736889 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-ee7e46ee-df2c-443c-af2e-ea57608d2de7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99be145c05a74cf081dceba537d46bf9'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:29.754933 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1278/2549] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:59:29 2022] DELETE /identity/v3/users/99be145c05a74cf081dceba537d46bf9 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:29.760250 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7467ab-515e-4021-a9c3-b2a84d0c3796 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:29.760748 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7467ab-515e-4021-a9c3-b2a84d0c3796 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:29.760865 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7467ab-515e-4021-a9c3-b2a84d0c3796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:29.779655 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-6b7467ab-515e-4021-a9c3-b2a84d0c3796 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:59:29.780868 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-6b7467ab-515e-4021-a9c3-b2a84d0c3796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:29.838694 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1272/2550] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:59:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:59:29.849030 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e46ee-df2c-443c-af2e-ea57608d2de7 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:29.850505 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-ee7e46ee-df2c-443c-af2e-ea57608d2de7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:29.911536 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e46ee-df2c-443c-af2e-ea57608d2de7 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:29.915832 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68cd68b-e03d-4fe7-9103-b347cc786439 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:29.916455 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cd68b-e03d-4fe7-9103-b347cc786439 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:29.916736 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cd68b-e03d-4fe7-9103-b347cc786439 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:29.917017 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cd68b-e03d-4fe7-9103-b347cc786439 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:29.926415 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68cd68b-e03d-4fe7-9103-b347cc786439 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:29.927854 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68cd68b-e03d-4fe7-9103-b347cc786439 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:29.938152 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1279/2551] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:59:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:59:30.214164 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7467ab-515e-4021-a9c3-b2a84d0c3796 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:30.215696 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-6b7467ab-515e-4021-a9c3-b2a84d0c3796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:30.228492 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7467ab-515e-4021-a9c3-b2a84d0c3796 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:30.233180 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967285a-3103-4bd8-a0a6-1ba965579885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:30.233929 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967285a-3103-4bd8-a0a6-1ba965579885 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:30.234273 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967285a-3103-4bd8-a0a6-1ba965579885 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:30.234608 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967285a-3103-4bd8-a0a6-1ba965579885 admin admin] PATH_INFO: `/v3/projects/5355372a696a4957b8c7efcd657e2c29` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:30.236477 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6967285a-3103-4bd8-a0a6-1ba965579885 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5355372a696a4957b8c7efcd657e2c29)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:30.238688 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6967285a-3103-4bd8-a0a6-1ba965579885 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:30.285802 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-6967285a-3103-4bd8-a0a6-1ba965579885 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5355372a696a4957b8c7efcd657e2c29', 'request_id': 'req-6967285a-3103-4bd8-a0a6-1ba965579885'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:30.293752 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-6967285a-3103-4bd8-a0a6-1ba965579885 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5355372a696a4957b8c7efcd657e2c29', 'request_id': 'req-6967285a-3103-4bd8-a0a6-1ba965579885'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:30.299426 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-6967285a-3103-4bd8-a0a6-1ba965579885 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5355372a696a4957b8c7efcd657e2c29', 'request_id': 'req-6967285a-3103-4bd8-a0a6-1ba965579885'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:30.310271 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-6967285a-3103-4bd8-a0a6-1ba965579885 admin admin] The token cache is being invalidate because project 5355372a696a4957b8c7efcd657e2c29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:59:30.310560 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-6967285a-3103-4bd8-a0a6-1ba965579885 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:30.311187 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-6967285a-3103-4bd8-a0a6-1ba965579885 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:30.312659 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1273/2552] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:59:30 2022] DELETE /identity/v3/projects/5355372a696a4957b8c7efcd657e2c29 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:30.316773 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68cd68b-e03d-4fe7-9103-b347cc786439 service neutron] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:30.326599 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68cd68b-e03d-4fe7-9103-b347cc786439 service neutron] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:30.382700 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60934e-fec8-4a91-8f84-bdd9fe55095a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:30.383255 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60934e-fec8-4a91-8f84-bdd9fe55095a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:30.383535 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60934e-fec8-4a91-8f84-bdd9fe55095a admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:30.383807 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60934e-fec8-4a91-8f84-bdd9fe55095a admin admin] PATH_INFO: `/v3/users/73a8f5c5b7624b3f905e1241fa6efd4c` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:30.384937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c60934e-fec8-4a91-8f84-bdd9fe55095a admin admin] RBAC: Authorizing `identity:delete_user(user_id=73a8f5c5b7624b3f905e1241fa6efd4c)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:30.386911 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c60934e-fec8-4a91-8f84-bdd9fe55095a admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:30.434024 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-5c60934e-fec8-4a91-8f84-bdd9fe55095a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73a8f5c5b7624b3f905e1241fa6efd4c'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:30.441450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-5c60934e-fec8-4a91-8f84-bdd9fe55095a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73a8f5c5b7624b3f905e1241fa6efd4c'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:30.449378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-5c60934e-fec8-4a91-8f84-bdd9fe55095a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73a8f5c5b7624b3f905e1241fa6efd4c'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:30.450034 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-5c60934e-fec8-4a91-8f84-bdd9fe55095a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73a8f5c5b7624b3f905e1241fa6efd4c'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:30.450629 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-5c60934e-fec8-4a91-8f84-bdd9fe55095a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73a8f5c5b7624b3f905e1241fa6efd4c'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:30.452778 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:30.471230 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1280/2553] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:59:30 2022] DELETE /identity/v3/users/73a8f5c5b7624b3f905e1241fa6efd4c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:30.751341 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967285a-3103-4bd8-a0a6-1ba965579885 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:30.752955 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-6967285a-3103-4bd8-a0a6-1ba965579885 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:30.770387 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967285a-3103-4bd8-a0a6-1ba965579885 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:30.775555 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:30.830977 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05063fd-08d9-4a27-ae96-d2bb7aad8a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:30.831610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05063fd-08d9-4a27-ae96-d2bb7aad8a38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:30.831890 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05063fd-08d9-4a27-ae96-d2bb7aad8a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:30.832164 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05063fd-08d9-4a27-ae96-d2bb7aad8a38 admin admin] PATH_INFO: `/v3/projects/b267403c20944393b1b71ce0a731954d` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:30.833361 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05063fd-08d9-4a27-ae96-d2bb7aad8a38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b267403c20944393b1b71ce0a731954d)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:30.835249 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05063fd-08d9-4a27-ae96-d2bb7aad8a38 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:30.883770 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f05063fd-08d9-4a27-ae96-d2bb7aad8a38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b267403c20944393b1b71ce0a731954d', 'request_id': 'req-f05063fd-08d9-4a27-ae96-d2bb7aad8a38'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:30.891209 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f05063fd-08d9-4a27-ae96-d2bb7aad8a38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b267403c20944393b1b71ce0a731954d', 'request_id': 'req-f05063fd-08d9-4a27-ae96-d2bb7aad8a38'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:30.897050 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f05063fd-08d9-4a27-ae96-d2bb7aad8a38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b267403c20944393b1b71ce0a731954d', 'request_id': 'req-f05063fd-08d9-4a27-ae96-d2bb7aad8a38'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:30.908814 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f05063fd-08d9-4a27-ae96-d2bb7aad8a38 admin admin] The token cache is being invalidate because project b267403c20944393b1b71ce0a731954d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:59:30.909109 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f05063fd-08d9-4a27-ae96-d2bb7aad8a38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:30.909789 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f05063fd-08d9-4a27-ae96-d2bb7aad8a38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:30.911307 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1274/2554] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:59:30 2022] DELETE /identity/v3/projects/b267403c20944393b1b71ce0a731954d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:30.915945 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60934e-fec8-4a91-8f84-bdd9fe55095a admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:30.917378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-5c60934e-fec8-4a91-8f84-bdd9fe55095a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:30.930681 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60934e-fec8-4a91-8f84-bdd9fe55095a admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:30.985183 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e282b991-80ed-4fa6-9043-c345bc83ddc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:30.985842 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282b991-80ed-4fa6-9043-c345bc83ddc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:30.986120 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282b991-80ed-4fa6-9043-c345bc83ddc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:30.986389 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282b991-80ed-4fa6-9043-c345bc83ddc4 admin admin] PATH_INFO: `/v3/users/33ac85866a6849db8c1766bafe6193bf` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:30.987484 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e282b991-80ed-4fa6-9043-c345bc83ddc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33ac85866a6849db8c1766bafe6193bf)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:30.989494 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e282b991-80ed-4fa6-9043-c345bc83ddc4 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:31.036585 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-e282b991-80ed-4fa6-9043-c345bc83ddc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33ac85866a6849db8c1766bafe6193bf'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:31.044861 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-e282b991-80ed-4fa6-9043-c345bc83ddc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33ac85866a6849db8c1766bafe6193bf'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:31.051965 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-e282b991-80ed-4fa6-9043-c345bc83ddc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33ac85866a6849db8c1766bafe6193bf'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:31.052639 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-e282b991-80ed-4fa6-9043-c345bc83ddc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33ac85866a6849db8c1766bafe6193bf'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:31.053243 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-e282b991-80ed-4fa6-9043-c345bc83ddc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33ac85866a6849db8c1766bafe6193bf'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:31.072746 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1281/2555] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:59:30 2022] DELETE /identity/v3/users/33ac85866a6849db8c1766bafe6193bf => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:31.345425 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05063fd-08d9-4a27-ae96-d2bb7aad8a38 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:31.346879 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-f05063fd-08d9-4a27-ae96-d2bb7aad8a38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:31.360855 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05063fd-08d9-4a27-ae96-d2bb7aad8a38 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:31.423820 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7023c11c-3676-4bc8-9145-e675ea9b8a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:31.424190 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023c11c-3676-4bc8-9145-e675ea9b8a36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:31.424500 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023c11c-3676-4bc8-9145-e675ea9b8a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:31.425396 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023c11c-3676-4bc8-9145-e675ea9b8a36 admin admin] PATH_INFO: `/v3/projects/af0a3b61594b4065b03a319dbb42625a` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:31.427063 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7023c11c-3676-4bc8-9145-e675ea9b8a36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af0a3b61594b4065b03a319dbb42625a)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:31.429115 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7023c11c-3676-4bc8-9145-e675ea9b8a36 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:31.471009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:31.479598 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-7023c11c-3676-4bc8-9145-e675ea9b8a36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af0a3b61594b4065b03a319dbb42625a', 'request_id': 'req-7023c11c-3676-4bc8-9145-e675ea9b8a36'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:31.486916 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-7023c11c-3676-4bc8-9145-e675ea9b8a36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af0a3b61594b4065b03a319dbb42625a', 'request_id': 'req-7023c11c-3676-4bc8-9145-e675ea9b8a36'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:31.492529 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-7023c11c-3676-4bc8-9145-e675ea9b8a36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af0a3b61594b4065b03a319dbb42625a', 'request_id': 'req-7023c11c-3676-4bc8-9145-e675ea9b8a36'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:31.503573 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-7023c11c-3676-4bc8-9145-e675ea9b8a36 admin admin] The token cache is being invalidate because project af0a3b61594b4065b03a319dbb42625a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:59:31.503859 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-7023c11c-3676-4bc8-9145-e675ea9b8a36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:31.504485 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-7023c11c-3676-4bc8-9145-e675ea9b8a36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:31.506004 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1275/2556] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:59:31 2022] DELETE /identity/v3/projects/af0a3b61594b4065b03a319dbb42625a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:31.512416 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebfbef-e66f-49d7-8294-9dbfc72be0b7 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:31.512709 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebfbef-e66f-49d7-8294-9dbfc72be0b7 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:31.512987 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebfbef-e66f-49d7-8294-9dbfc72be0b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:31.532752 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-acebfbef-e66f-49d7-8294-9dbfc72be0b7 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:59:31.534007 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-acebfbef-e66f-49d7-8294-9dbfc72be0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:31.592101 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1282/2557] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:59:31.600395 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca720da-2f48-483a-ad7a-4575b69aef83 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:31.600682 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca720da-2f48-483a-ad7a-4575b69aef83 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:31.600957 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca720da-2f48-483a-ad7a-4575b69aef83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:31.620305 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-6ca720da-2f48-483a-ad7a-4575b69aef83 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:59:31.621543 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-6ca720da-2f48-483a-ad7a-4575b69aef83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:31.629670 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1276/2558] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:59:31.635209 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebfbef-e66f-49d7-8294-9dbfc72be0b7 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:31.640032 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:31.646494 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebfbef-e66f-49d7-8294-9dbfc72be0b7 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:31.652814 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527669a-adee-4f5e-b100-4c99db617ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:31.653457 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527669a-adee-4f5e-b100-4c99db617ed7 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:31.653748 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527669a-adee-4f5e-b100-4c99db617ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:31.654045 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527669a-adee-4f5e-b100-4c99db617ed7 admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:31.654685 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d527669a-adee-4f5e-b100-4c99db617ed7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:59:31.654988 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d527669a-adee-4f5e-b100-4c99db617ed7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:31.656592 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d527669a-adee-4f5e-b100-4c99db617ed7 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:31.668051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1283/2559] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:59:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:59:31.672384 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca720da-2f48-483a-ad7a-4575b69aef83 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:31.681903 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca720da-2f48-483a-ad7a-4575b69aef83 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:31.686284 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7615ab16-e790-4145-99e9-5e3eb7b159d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:31.686891 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615ab16-e790-4145-99e9-5e3eb7b159d3 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:31.687195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615ab16-e790-4145-99e9-5e3eb7b159d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:31.687478 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615ab16-e790-4145-99e9-5e3eb7b159d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:31.688128 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7615ab16-e790-4145-99e9-5e3eb7b159d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:31.690194 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7615ab16-e790-4145-99e9-5e3eb7b159d3 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:31.718155 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1277/2560] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:59:31 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:59:31.722408 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527669a-adee-4f5e-b100-4c99db617ed7 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:31.733481 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527669a-adee-4f5e-b100-4c99db617ed7 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:31.793352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc534b8-5084-40fe-b6d6-3ea1ee6eb122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:31.793977 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc534b8-5084-40fe-b6d6-3ea1ee6eb122 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:31.794262 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc534b8-5084-40fe-b6d6-3ea1ee6eb122 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:31.794541 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc534b8-5084-40fe-b6d6-3ea1ee6eb122 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:31.795159 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc534b8-5084-40fe-b6d6-3ea1ee6eb122 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:31.797103 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc534b8-5084-40fe-b6d6-3ea1ee6eb122 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:31.818466 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1284/2561] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:59:31 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:59:31.822698 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7615ab16-e790-4145-99e9-5e3eb7b159d3 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:31.833051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7615ab16-e790-4145-99e9-5e3eb7b159d3 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:31.838273 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5295ce6-8c98-406c-a72a-3ae96fd6054a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:31.838868 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5295ce6-8c98-406c-a72a-3ae96fd6054a admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:31.839144 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5295ce6-8c98-406c-a72a-3ae96fd6054a admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:31.839411 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5295ce6-8c98-406c-a72a-3ae96fd6054a admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:31.840066 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5295ce6-8c98-406c-a72a-3ae96fd6054a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:31.841692 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5295ce6-8c98-406c-a72a-3ae96fd6054a admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:31.852431 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1278/2562] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 22:59:31.857478 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc534b8-5084-40fe-b6d6-3ea1ee6eb122 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:31.868515 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc534b8-5084-40fe-b6d6-3ea1ee6eb122 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:31.873544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c2f73a-78a0-4805-92ac-d4d4a68c7fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:31.874143 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2f73a-78a0-4805-92ac-d4d4a68c7fd4 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:31.874487 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2f73a-78a0-4805-92ac-d4d4a68c7fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:31.874744 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2f73a-78a0-4805-92ac-d4d4a68c7fd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:31.875386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c2f73a-78a0-4805-92ac-d4d4a68c7fd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:31.876976 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c2f73a-78a0-4805-92ac-d4d4a68c7fd4 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:31.887058 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1285/2563] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:59:31.891297 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5295ce6-8c98-406c-a72a-3ae96fd6054a admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:31.902835 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5295ce6-8c98-406c-a72a-3ae96fd6054a admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:31.907583 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6366b68-b9d3-4256-84f7-9c0014166ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:31.908267 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6366b68-b9d3-4256-84f7-9c0014166ee4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:31.908546 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6366b68-b9d3-4256-84f7-9c0014166ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:31.908820 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6366b68-b9d3-4256-84f7-9c0014166ee4 admin admin] PATH_INFO: `/v3/projects/b6fc22b53ec8411692476be08f777c3b/users/4cf93918535c42f6b2d51f2578f3e24a/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:31.925246 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6366b68-b9d3-4256-84f7-9c0014166ee4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6fc22b53ec8411692476be08f777c3b, user_id=4cf93918535c42f6b2d51f2578f3e24a, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:31.927974 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6366b68-b9d3-4256-84f7-9c0014166ee4 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:31.941582 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1279/2564] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:59:31 2022] PUT /identity/v3/projects/b6fc22b53ec8411692476be08f777c3b/users/4cf93918535c42f6b2d51f2578f3e24a/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:31.954443 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c31d74-b229-4665-8040-df3ded2b75d7 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:31.954942 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c31d74-b229-4665-8040-df3ded2b75d7 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:31.955064 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c31d74-b229-4665-8040-df3ded2b75d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:31.982712 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-10c31d74-b229-4665-8040-df3ded2b75d7 None None] MFA Rules not processed for user `4cf93918535c42f6b2d51f2578f3e24a`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:59:31.983851 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-10c31d74-b229-4665-8040-df3ded2b75d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:32.053533 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1286/2565] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:59:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 22:59:32.058595 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6366b68-b9d3-4256-84f7-9c0014166ee4 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:32.122173 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6366b68-b9d3-4256-84f7-9c0014166ee4 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:32.127133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d17944-4c07-4f28-966a-eb1837ba11bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:32.127755 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d17944-4c07-4f28-966a-eb1837ba11bb admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:32.128025 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d17944-4c07-4f28-966a-eb1837ba11bb admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:32.128300 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d17944-4c07-4f28-966a-eb1837ba11bb admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:32.128913 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d17944-4c07-4f28-966a-eb1837ba11bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:32.130864 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d17944-4c07-4f28-966a-eb1837ba11bb admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:32.157652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1280/2566] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:59:32 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:59:32.161808 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c31d74-b229-4665-8040-df3ded2b75d7 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:32.173591 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c31d74-b229-4665-8040-df3ded2b75d7 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:32.224226 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:32.231304 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1d486-719b-472b-bab5-53d21a7ed801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:32.231930 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1d486-719b-472b-bab5-53d21a7ed801 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:32.232377 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1d486-719b-472b-bab5-53d21a7ed801 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:32.232652 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1d486-719b-472b-bab5-53d21a7ed801 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:32.233357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd1d486-719b-472b-bab5-53d21a7ed801 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:32.235226 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd1d486-719b-472b-bab5-53d21a7ed801 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:32.256943 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1287/2567] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:59:32 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:59:32.261777 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d17944-4c07-4f28-966a-eb1837ba11bb admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:32.272500 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d17944-4c07-4f28-966a-eb1837ba11bb admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:32.278132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803faa3-7ac1-41b1-866e-ece5e477f03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:32.278736 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803faa3-7ac1-41b1-866e-ece5e477f03b admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:32.279021 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803faa3-7ac1-41b1-866e-ece5e477f03b admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:32.279301 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803faa3-7ac1-41b1-866e-ece5e477f03b admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:32.279942 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d803faa3-7ac1-41b1-866e-ece5e477f03b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:32.281626 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d803faa3-7ac1-41b1-866e-ece5e477f03b admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:32.292245 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1281/2568] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:59:32.296372 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1d486-719b-472b-bab5-53d21a7ed801 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:32.306344 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1d486-719b-472b-bab5-53d21a7ed801 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:32.311179 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa54bf-e7d9-4012-9ffc-c3c4154e905a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:32.311909 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa54bf-e7d9-4012-9ffc-c3c4154e905a admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:32.312184 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa54bf-e7d9-4012-9ffc-c3c4154e905a admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:32.312454 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa54bf-e7d9-4012-9ffc-c3c4154e905a admin admin] PATH_INFO: `/v3/projects/e257d793529f4c0cbaab7f0180342709/users/ae0761824def414abcf714b9586d69ed/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:32.327803 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa54bf-e7d9-4012-9ffc-c3c4154e905a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e257d793529f4c0cbaab7f0180342709, user_id=ae0761824def414abcf714b9586d69ed, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:32.330512 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa54bf-e7d9-4012-9ffc-c3c4154e905a admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:32.343997 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1288/2569] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:59:32 2022] PUT /identity/v3/projects/e257d793529f4c0cbaab7f0180342709/users/ae0761824def414abcf714b9586d69ed/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:32.348036 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803faa3-7ac1-41b1-866e-ece5e477f03b admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:32.358265 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803faa3-7ac1-41b1-866e-ece5e477f03b admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:32.391445 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:32.417630 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba5ee6-cbd7-4524-9f54-98289fc14a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:32.418252 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba5ee6-cbd7-4524-9f54-98289fc14a0f admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:32.418533 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba5ee6-cbd7-4524-9f54-98289fc14a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:32.418807 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba5ee6-cbd7-4524-9f54-98289fc14a0f admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:32.419442 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba5ee6-cbd7-4524-9f54-98289fc14a0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:32.421050 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba5ee6-cbd7-4524-9f54-98289fc14a0f admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:32.431675 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1282/2570] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:59:32.435819 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa54bf-e7d9-4012-9ffc-c3c4154e905a admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:32.446887 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa54bf-e7d9-4012-9ffc-c3c4154e905a admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:32.451572 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe21be51-240c-491e-898b-aee605f4a5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:32.452217 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21be51-240c-491e-898b-aee605f4a5e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:32.452496 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21be51-240c-491e-898b-aee605f4a5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:32.452766 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21be51-240c-491e-898b-aee605f4a5e3 admin admin] PATH_INFO: `/v3/projects/e257d793529f4c0cbaab7f0180342709/users/ae0761824def414abcf714b9586d69ed/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:32.454813 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe21be51-240c-491e-898b-aee605f4a5e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e257d793529f4c0cbaab7f0180342709, user_id=ae0761824def414abcf714b9586d69ed, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:32.457632 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe21be51-240c-491e-898b-aee605f4a5e3 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:32.468630 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1289/2571] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:59:32 2022] PUT /identity/v3/projects/e257d793529f4c0cbaab7f0180342709/users/ae0761824def414abcf714b9586d69ed/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:32.480216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ac8ef-d110-45dc-b08b-756a9a861c11 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:32.480507 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ac8ef-d110-45dc-b08b-756a9a861c11 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:32.480785 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ac8ef-d110-45dc-b08b-756a9a861c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:32.509512 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-ac1ac8ef-d110-45dc-b08b-756a9a861c11 None None] MFA Rules not processed for user `ae0761824def414abcf714b9586d69ed`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:59:32.511307 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-ac1ac8ef-d110-45dc-b08b-756a9a861c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:32.578525 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1283/2572] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:59:32 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:59:32.589325 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe21be51-240c-491e-898b-aee605f4a5e3 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:32.591037 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-fe21be51-240c-491e-898b-aee605f4a5e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:32.647722 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe21be51-240c-491e-898b-aee605f4a5e3 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:32.651480 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d3c42-7f3d-454a-b84f-f7b87d6600cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:32.652075 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d3c42-7f3d-454a-b84f-f7b87d6600cb service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:32.652357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d3c42-7f3d-454a-b84f-f7b87d6600cb service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:32.652627 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d3c42-7f3d-454a-b84f-f7b87d6600cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:32.661576 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d3c42-7f3d-454a-b84f-f7b87d6600cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:32.663010 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d3c42-7f3d-454a-b84f-f7b87d6600cb service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:32.672470 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1290/2573] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:59:32 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:59:33.254278 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1ac8ef-d110-45dc-b08b-756a9a861c11 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:33.315545 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1ac8ef-d110-45dc-b08b-756a9a861c11 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:33.320681 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57abb84-3ca8-4cb2-aaa7-61feb46b2e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:33.321245 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57abb84-3ca8-4cb2-aaa7-61feb46b2e2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:33.321549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57abb84-3ca8-4cb2-aaa7-61feb46b2e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:33.321819 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57abb84-3ca8-4cb2-aaa7-61feb46b2e2e admin admin] PATH_INFO: `/v3/users/4cf93918535c42f6b2d51f2578f3e24a` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:33.322946 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57abb84-3ca8-4cb2-aaa7-61feb46b2e2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cf93918535c42f6b2d51f2578f3e24a)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:33.324874 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57abb84-3ca8-4cb2-aaa7-61feb46b2e2e admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:33.373359 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-d57abb84-3ca8-4cb2-aaa7-61feb46b2e2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cf93918535c42f6b2d51f2578f3e24a'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:33.380479 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-d57abb84-3ca8-4cb2-aaa7-61feb46b2e2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cf93918535c42f6b2d51f2578f3e24a'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:33.385720 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:33.394706 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-d57abb84-3ca8-4cb2-aaa7-61feb46b2e2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cf93918535c42f6b2d51f2578f3e24a'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:33.395405 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-d57abb84-3ca8-4cb2-aaa7-61feb46b2e2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cf93918535c42f6b2d51f2578f3e24a'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:33.396009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-d57abb84-3ca8-4cb2-aaa7-61feb46b2e2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cf93918535c42f6b2d51f2578f3e24a'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:33.408856 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1284/2574] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:59:33 2022] DELETE /identity/v3/users/4cf93918535c42f6b2d51f2578f3e24a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:33.413411 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9dbc97-074d-4225-9b9a-d00c2aa1db08 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:33.413919 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9dbc97-074d-4225-9b9a-d00c2aa1db08 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:33.414096 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9dbc97-074d-4225-9b9a-d00c2aa1db08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:33.433306 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-2c9dbc97-074d-4225-9b9a-d00c2aa1db08 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:59:33.434538 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-2c9dbc97-074d-4225-9b9a-d00c2aa1db08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:33.493434 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1291/2575] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:59:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:59:33.503430 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57abb84-3ca8-4cb2-aaa7-61feb46b2e2e admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:33.504839 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-d57abb84-3ca8-4cb2-aaa7-61feb46b2e2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:33.561555 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57abb84-3ca8-4cb2-aaa7-61feb46b2e2e admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:33.565359 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a8b54-3f7d-4c91-92ff-92c857f5c951 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:33.565972 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a8b54-3f7d-4c91-92ff-92c857f5c951 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:33.566245 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a8b54-3f7d-4c91-92ff-92c857f5c951 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:33.566511 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a8b54-3f7d-4c91-92ff-92c857f5c951 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:33.576370 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a8b54-3f7d-4c91-92ff-92c857f5c951 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:33.577770 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a8b54-3f7d-4c91-92ff-92c857f5c951 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:33.588505 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1285/2576] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:59:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:59:34.122710 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9dbc97-074d-4225-9b9a-d00c2aa1db08 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:34.124441 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-2c9dbc97-074d-4225-9b9a-d00c2aa1db08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:34.138091 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9dbc97-074d-4225-9b9a-d00c2aa1db08 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:34.143026 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb16afbc-a550-469d-bf70-642c9b983db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:34.143673 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb16afbc-a550-469d-bf70-642c9b983db1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:34.143959 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb16afbc-a550-469d-bf70-642c9b983db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:34.144233 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb16afbc-a550-469d-bf70-642c9b983db1 admin admin] PATH_INFO: `/v3/projects/b6fc22b53ec8411692476be08f777c3b` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:34.145405 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb16afbc-a550-469d-bf70-642c9b983db1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6fc22b53ec8411692476be08f777c3b)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:34.147364 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb16afbc-a550-469d-bf70-642c9b983db1 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:34.194307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:34.196661 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-eb16afbc-a550-469d-bf70-642c9b983db1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6fc22b53ec8411692476be08f777c3b', 'request_id': 'req-eb16afbc-a550-469d-bf70-642c9b983db1'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:34.203764 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-eb16afbc-a550-469d-bf70-642c9b983db1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6fc22b53ec8411692476be08f777c3b', 'request_id': 'req-eb16afbc-a550-469d-bf70-642c9b983db1'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:34.211769 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-eb16afbc-a550-469d-bf70-642c9b983db1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6fc22b53ec8411692476be08f777c3b', 'request_id': 'req-eb16afbc-a550-469d-bf70-642c9b983db1'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:34.223086 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-eb16afbc-a550-469d-bf70-642c9b983db1 admin admin] The token cache is being invalidate because project b6fc22b53ec8411692476be08f777c3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:59:34.223372 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-eb16afbc-a550-469d-bf70-642c9b983db1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:34.223991 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-eb16afbc-a550-469d-bf70-642c9b983db1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:34.225485 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1292/2577] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:59:34 2022] DELETE /identity/v3/projects/b6fc22b53ec8411692476be08f777c3b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:34.229841 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a8b54-3f7d-4c91-92ff-92c857f5c951 service neutron] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:34.239371 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a8b54-3f7d-4c91-92ff-92c857f5c951 service neutron] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:34.295670 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1099d873-1625-482e-8e95-094837ef7e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:34.296323 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099d873-1625-482e-8e95-094837ef7e19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:34.296486 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099d873-1625-482e-8e95-094837ef7e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:34.296759 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099d873-1625-482e-8e95-094837ef7e19 admin admin] PATH_INFO: `/v3/users/ae0761824def414abcf714b9586d69ed` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:34.297874 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1099d873-1625-482e-8e95-094837ef7e19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae0761824def414abcf714b9586d69ed)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:34.299796 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1099d873-1625-482e-8e95-094837ef7e19 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:34.344970 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-1099d873-1625-482e-8e95-094837ef7e19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae0761824def414abcf714b9586d69ed'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:34.352453 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-1099d873-1625-482e-8e95-094837ef7e19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae0761824def414abcf714b9586d69ed'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:34.364067 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-1099d873-1625-482e-8e95-094837ef7e19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae0761824def414abcf714b9586d69ed'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:34.364739 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-1099d873-1625-482e-8e95-094837ef7e19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae0761824def414abcf714b9586d69ed'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:34.365377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-1099d873-1625-482e-8e95-094837ef7e19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae0761824def414abcf714b9586d69ed'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:34.375314 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:34.380375 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1286/2578] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:59:34 2022] DELETE /identity/v3/users/ae0761824def414abcf714b9586d69ed => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:34.656456 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb16afbc-a550-469d-bf70-642c9b983db1 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:34.657961 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-eb16afbc-a550-469d-bf70-642c9b983db1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:34.673242 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb16afbc-a550-469d-bf70-642c9b983db1 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:34.730015 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f2a1a-d7a4-45aa-bdd0-1e82e4e253f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:34.730651 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f2a1a-d7a4-45aa-bdd0-1e82e4e253f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:34.730936 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f2a1a-d7a4-45aa-bdd0-1e82e4e253f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:34.731214 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f2a1a-d7a4-45aa-bdd0-1e82e4e253f7 admin admin] PATH_INFO: `/v3/projects/e257d793529f4c0cbaab7f0180342709` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:34.732340 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f2a1a-d7a4-45aa-bdd0-1e82e4e253f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e257d793529f4c0cbaab7f0180342709)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:34.734301 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f2a1a-d7a4-45aa-bdd0-1e82e4e253f7 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:34.780652 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-e65f2a1a-d7a4-45aa-bdd0-1e82e4e253f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e257d793529f4c0cbaab7f0180342709', 'request_id': 'req-e65f2a1a-d7a4-45aa-bdd0-1e82e4e253f7'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:34.786222 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-e65f2a1a-d7a4-45aa-bdd0-1e82e4e253f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e257d793529f4c0cbaab7f0180342709', 'request_id': 'req-e65f2a1a-d7a4-45aa-bdd0-1e82e4e253f7'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:34.793644 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-e65f2a1a-d7a4-45aa-bdd0-1e82e4e253f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e257d793529f4c0cbaab7f0180342709', 'request_id': 'req-e65f2a1a-d7a4-45aa-bdd0-1e82e4e253f7'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:34.805386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-e65f2a1a-d7a4-45aa-bdd0-1e82e4e253f7 admin admin] The token cache is being invalidate because project e257d793529f4c0cbaab7f0180342709 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:59:34.805671 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-e65f2a1a-d7a4-45aa-bdd0-1e82e4e253f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:34.806304 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-e65f2a1a-d7a4-45aa-bdd0-1e82e4e253f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:34.807768 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1293/2579] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:59:34 2022] DELETE /identity/v3/projects/e257d793529f4c0cbaab7f0180342709 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:34.812635 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db7877-c58e-4854-88b1-a6528435c6fb None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:34.812919 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db7877-c58e-4854-88b1-a6528435c6fb None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:34.813193 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db7877-c58e-4854-88b1-a6528435c6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:34.833399 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-65db7877-c58e-4854-88b1-a6528435c6fb None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:59:34.834558 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-65db7877-c58e-4854-88b1-a6528435c6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:34.894787 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1287/2580] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:59:34.904079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854bf2b-d4cd-4561-8f64-b92e7f225f8b None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:34.904363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854bf2b-d4cd-4561-8f64-b92e7f225f8b None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:34.904636 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854bf2b-d4cd-4561-8f64-b92e7f225f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:34.924572 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-e854bf2b-d4cd-4561-8f64-b92e7f225f8b None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:59:34.925802 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-e854bf2b-d4cd-4561-8f64-b92e7f225f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:34.933780 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1294/2581] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:59:34.938009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65db7877-c58e-4854-88b1-a6528435c6fb None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:34.947969 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65db7877-c58e-4854-88b1-a6528435c6fb None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:34.952547 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f48ab1-adb5-467f-8196-2a5c48363c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:34.953146 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f48ab1-adb5-467f-8196-2a5c48363c98 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:34.953465 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f48ab1-adb5-467f-8196-2a5c48363c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:34.953735 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f48ab1-adb5-467f-8196-2a5c48363c98 admin admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:34.954375 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f48ab1-adb5-467f-8196-2a5c48363c98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:59:34.954674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f48ab1-adb5-467f-8196-2a5c48363c98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:34.956265 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f48ab1-adb5-467f-8196-2a5c48363c98 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:34.968126 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1288/2582] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:59:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:59:34.972304 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854bf2b-d4cd-4561-8f64-b92e7f225f8b None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:34.982425 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854bf2b-d4cd-4561-8f64-b92e7f225f8b None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:34.987058 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53d4ac-8d94-486c-9ec1-fae99a69fac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:34.987650 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53d4ac-8d94-486c-9ec1-fae99a69fac4 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:34.987927 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53d4ac-8d94-486c-9ec1-fae99a69fac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:34.988196 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53d4ac-8d94-486c-9ec1-fae99a69fac4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:34.988873 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf53d4ac-8d94-486c-9ec1-fae99a69fac4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:34.990891 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf53d4ac-8d94-486c-9ec1-fae99a69fac4 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:35.017452 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1295/2583] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:59:34 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:59:35.021679 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f48ab1-adb5-467f-8196-2a5c48363c98 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:35.032432 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f48ab1-adb5-467f-8196-2a5c48363c98 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:35.089125 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b6a5d0-e1c2-4403-848e-b8e388910b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:35.089754 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6a5d0-e1c2-4403-848e-b8e388910b6b admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:35.090039 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6a5d0-e1c2-4403-848e-b8e388910b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:35.090315 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6a5d0-e1c2-4403-848e-b8e388910b6b admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:35.091211 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b6a5d0-e1c2-4403-848e-b8e388910b6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:35.093125 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b6a5d0-e1c2-4403-848e-b8e388910b6b admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:35.115403 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1289/2584] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:59:35 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:59:35.119459 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53d4ac-8d94-486c-9ec1-fae99a69fac4 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:35.129859 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53d4ac-8d94-486c-9ec1-fae99a69fac4 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:35.136077 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb304de3-3bcc-4fe0-a44f-ec9d80e306c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:35.136676 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb304de3-3bcc-4fe0-a44f-ec9d80e306c1 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:35.136953 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb304de3-3bcc-4fe0-a44f-ec9d80e306c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:35.137227 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb304de3-3bcc-4fe0-a44f-ec9d80e306c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:35.137899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb304de3-3bcc-4fe0-a44f-ec9d80e306c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:35.139489 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb304de3-3bcc-4fe0-a44f-ec9d80e306c1 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:35.150015 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1296/2585] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:59:35.154130 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b6a5d0-e1c2-4403-848e-b8e388910b6b admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:35.166201 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b6a5d0-e1c2-4403-848e-b8e388910b6b admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:35.171049 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c48f7-a6c4-4916-a1ee-e6480f7e42f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:35.171766 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c48f7-a6c4-4916-a1ee-e6480f7e42f0 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:35.172048 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c48f7-a6c4-4916-a1ee-e6480f7e42f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:35.172323 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c48f7-a6c4-4916-a1ee-e6480f7e42f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:35.172962 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5c48f7-a6c4-4916-a1ee-e6480f7e42f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:35.174610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5c48f7-a6c4-4916-a1ee-e6480f7e42f0 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:35.186011 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1290/2586] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:59:35.190186 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb304de3-3bcc-4fe0-a44f-ec9d80e306c1 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:35.206707 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb304de3-3bcc-4fe0-a44f-ec9d80e306c1 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:35.212510 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c2019-85ed-442a-bfa8-db7b3581107b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:35.213168 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c2019-85ed-442a-bfa8-db7b3581107b admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:35.213483 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c2019-85ed-442a-bfa8-db7b3581107b admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:35.213762 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c2019-85ed-442a-bfa8-db7b3581107b admin admin] PATH_INFO: `/v3/projects/678f3ac94c414898ad065452966e70a5/users/7a3c1c4260c940928edb80336520d4d6/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:35.235648 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001c2019-85ed-442a-bfa8-db7b3581107b admin admin] RBAC: Authorizing `identity:create_grant(project_id=678f3ac94c414898ad065452966e70a5, user_id=7a3c1c4260c940928edb80336520d4d6, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:35.238548 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001c2019-85ed-442a-bfa8-db7b3581107b admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:35.252086 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1297/2587] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:59:35 2022] PUT /identity/v3/projects/678f3ac94c414898ad065452966e70a5/users/7a3c1c4260c940928edb80336520d4d6/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:35.263445 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb4ecc-25ed-4555-b509-667be2a1802d None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:35.263735 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb4ecc-25ed-4555-b509-667be2a1802d None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:35.264010 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb4ecc-25ed-4555-b509-667be2a1802d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:35.293237 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-c8eb4ecc-25ed-4555-b509-667be2a1802d None None] MFA Rules not processed for user `7a3c1c4260c940928edb80336520d4d6`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:59:35.294542 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-c8eb4ecc-25ed-4555-b509-667be2a1802d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:35.349579 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:35.359710 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1291/2588] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:59:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:59:35.364222 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c2019-85ed-442a-bfa8-db7b3581107b admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:35.368923 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:35.429779 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c2019-85ed-442a-bfa8-db7b3581107b admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:35.434717 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da1b227-959a-4eac-9546-a6fd641f238b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:35.435333 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1b227-959a-4eac-9546-a6fd641f238b admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:35.435612 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1b227-959a-4eac-9546-a6fd641f238b admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:35.435882 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1b227-959a-4eac-9546-a6fd641f238b admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:35.436540 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da1b227-959a-4eac-9546-a6fd641f238b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:35.438492 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da1b227-959a-4eac-9546-a6fd641f238b admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:35.466013 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1298/2589] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:59:35 2022] POST /identity/v3/projects => generated 381 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:59:35.470446 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eb4ecc-25ed-4555-b509-667be2a1802d None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:35.482585 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eb4ecc-25ed-4555-b509-667be2a1802d None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:35.538191 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699b2804-7401-4c93-bf55-ba80c9a3b87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:35.539255 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b2804-7401-4c93-bf55-ba80c9a3b87f admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:35.539541 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b2804-7401-4c93-bf55-ba80c9a3b87f admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:35.539815 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b2804-7401-4c93-bf55-ba80c9a3b87f admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:35.540456 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699b2804-7401-4c93-bf55-ba80c9a3b87f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:35.543153 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699b2804-7401-4c93-bf55-ba80c9a3b87f admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:35.566801 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1292/2590] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:59:35 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:59:35.571763 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da1b227-959a-4eac-9546-a6fd641f238b admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:35.582178 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da1b227-959a-4eac-9546-a6fd641f238b admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:35.587187 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c8d03-ca5e-4638-9853-e4b6d36406fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:35.587795 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c8d03-ca5e-4638-9853-e4b6d36406fc admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:35.588081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c8d03-ca5e-4638-9853-e4b6d36406fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:35.588358 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c8d03-ca5e-4638-9853-e4b6d36406fc admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:35.589031 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906c8d03-ca5e-4638-9853-e4b6d36406fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:35.590684 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906c8d03-ca5e-4638-9853-e4b6d36406fc admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:35.601167 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1299/2591] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 22:59:35.605268 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699b2804-7401-4c93-bf55-ba80c9a3b87f admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:35.615220 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699b2804-7401-4c93-bf55-ba80c9a3b87f admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:35.620695 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be61af-6faf-4d20-9390-9ea1f3988dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:35.621387 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be61af-6faf-4d20-9390-9ea1f3988dd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:35.621674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be61af-6faf-4d20-9390-9ea1f3988dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:35.621951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be61af-6faf-4d20-9390-9ea1f3988dd5 admin admin] PATH_INFO: `/v3/projects/22f59c9c1d91405dab44aaff4075609e/users/379a47fda05842de905933d9e360af49/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:35.639470 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be61af-6faf-4d20-9390-9ea1f3988dd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22f59c9c1d91405dab44aaff4075609e, user_id=379a47fda05842de905933d9e360af49, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:35.642193 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be61af-6faf-4d20-9390-9ea1f3988dd5 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:35.654571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1293/2592] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:59:35 2022] PUT /identity/v3/projects/22f59c9c1d91405dab44aaff4075609e/users/379a47fda05842de905933d9e360af49/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:35.658566 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c8d03-ca5e-4638-9853-e4b6d36406fc admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:35.668989 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c8d03-ca5e-4638-9853-e4b6d36406fc admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:35.726185 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543523af-fd44-40b5-843d-8cdfa84e753f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:35.726855 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543523af-fd44-40b5-843d-8cdfa84e753f admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:35.727132 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543523af-fd44-40b5-843d-8cdfa84e753f admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:35.727406 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543523af-fd44-40b5-843d-8cdfa84e753f admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:35.728038 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543523af-fd44-40b5-843d-8cdfa84e753f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:35.729677 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543523af-fd44-40b5-843d-8cdfa84e753f admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:35.739546 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1300/2593] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:59:35.744630 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be61af-6faf-4d20-9390-9ea1f3988dd5 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:35.755222 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be61af-6faf-4d20-9390-9ea1f3988dd5 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:35.760698 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac4177-d4f0-4c7a-8440-318c857cf229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:35.761378 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac4177-d4f0-4c7a-8440-318c857cf229 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:35.761669 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac4177-d4f0-4c7a-8440-318c857cf229 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:35.761947 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac4177-d4f0-4c7a-8440-318c857cf229 admin admin] PATH_INFO: `/v3/projects/22f59c9c1d91405dab44aaff4075609e/users/379a47fda05842de905933d9e360af49/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:35.763873 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac4177-d4f0-4c7a-8440-318c857cf229 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22f59c9c1d91405dab44aaff4075609e, user_id=379a47fda05842de905933d9e360af49, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:35.766581 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac4177-d4f0-4c7a-8440-318c857cf229 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:35.777629 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1294/2594] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:59:35 2022] PUT /identity/v3/projects/22f59c9c1d91405dab44aaff4075609e/users/379a47fda05842de905933d9e360af49/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:35.789362 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21e694-1069-4b8c-bcd8-1719406649f1 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:35.789639 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21e694-1069-4b8c-bcd8-1719406649f1 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:35.789941 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21e694-1069-4b8c-bcd8-1719406649f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:35.817871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-6e21e694-1069-4b8c-bcd8-1719406649f1 None None] MFA Rules not processed for user `379a47fda05842de905933d9e360af49`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:59:35.819129 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-6e21e694-1069-4b8c-bcd8-1719406649f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:35.884610 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:35.890924 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1301/2595] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:59:35 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:59:35.901978 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac4177-d4f0-4c7a-8440-318c857cf229 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:35.903825 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-91ac4177-d4f0-4c7a-8440-318c857cf229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:35.963562 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac4177-d4f0-4c7a-8440-318c857cf229 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:35.967228 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17358654-d6b2-4a75-89d3-834d78f6e079 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:35.967826 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17358654-d6b2-4a75-89d3-834d78f6e079 service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:35.968102 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17358654-d6b2-4a75-89d3-834d78f6e079 service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:35.968372 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17358654-d6b2-4a75-89d3-834d78f6e079 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:35.977331 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17358654-d6b2-4a75-89d3-834d78f6e079 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:35.978768 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17358654-d6b2-4a75-89d3-834d78f6e079 service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:35.988297 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1295/2596] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:59:35 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:59:36.359057 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e21e694-1069-4b8c-bcd8-1719406649f1 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:36.420820 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e21e694-1069-4b8c-bcd8-1719406649f1 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:36.425297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbeb612-1a8b-476d-8f39-edfc2f10e9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:36.425986 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbeb612-1a8b-476d-8f39-edfc2f10e9dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:36.426268 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbeb612-1a8b-476d-8f39-edfc2f10e9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:36.426533 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbeb612-1a8b-476d-8f39-edfc2f10e9dd admin admin] PATH_INFO: `/v3/users/7a3c1c4260c940928edb80336520d4d6` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:36.427695 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbeb612-1a8b-476d-8f39-edfc2f10e9dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a3c1c4260c940928edb80336520d4d6)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:36.429869 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbeb612-1a8b-476d-8f39-edfc2f10e9dd admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:36.485622 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-8bbeb612-1a8b-476d-8f39-edfc2f10e9dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a3c1c4260c940928edb80336520d4d6'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:36.493269 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-8bbeb612-1a8b-476d-8f39-edfc2f10e9dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a3c1c4260c940928edb80336520d4d6'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:36.500394 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-8bbeb612-1a8b-476d-8f39-edfc2f10e9dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a3c1c4260c940928edb80336520d4d6'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:36.501039 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-8bbeb612-1a8b-476d-8f39-edfc2f10e9dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a3c1c4260c940928edb80336520d4d6'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:36.501679 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-8bbeb612-1a8b-476d-8f39-edfc2f10e9dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a3c1c4260c940928edb80336520d4d6'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:36.518377 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1302/2597] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:59:36 2022] DELETE /identity/v3/users/7a3c1c4260c940928edb80336520d4d6 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:36.523401 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608606f-56af-415f-b3df-3c555860a7f6 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:36.523958 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608606f-56af-415f-b3df-3c555860a7f6 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:36.523958 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608606f-56af-415f-b3df-3c555860a7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:36.543088 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-4608606f-56af-415f-b3df-3c555860a7f6 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:59:36.544284 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-4608606f-56af-415f-b3df-3c555860a7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:36.588153 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:36.603334 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1296/2598] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:59:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:59:36.613019 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbeb612-1a8b-476d-8f39-edfc2f10e9dd admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:36.614437 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-8bbeb612-1a8b-476d-8f39-edfc2f10e9dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:36.671646 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbeb612-1a8b-476d-8f39-edfc2f10e9dd admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:36.675623 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf5fb0-af5b-4c46-93e8-2334aa8398a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:36.676233 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf5fb0-af5b-4c46-93e8-2334aa8398a1 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:36.676555 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf5fb0-af5b-4c46-93e8-2334aa8398a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:36.676826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf5fb0-af5b-4c46-93e8-2334aa8398a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:36.685780 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf5fb0-af5b-4c46-93e8-2334aa8398a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:36.687203 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf5fb0-af5b-4c46-93e8-2334aa8398a1 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:36.697479 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1303/2599] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:59:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:59:36.966151 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4608606f-56af-415f-b3df-3c555860a7f6 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:36.967781 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-4608606f-56af-415f-b3df-3c555860a7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:36.980612 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4608606f-56af-415f-b3df-3c555860a7f6 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:36.985418 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18bcc2e-b0fc-4875-ab9e-d8d6eb7132cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:36.986062 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18bcc2e-b0fc-4875-ab9e-d8d6eb7132cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:36.986339 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18bcc2e-b0fc-4875-ab9e-d8d6eb7132cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:36.986610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18bcc2e-b0fc-4875-ab9e-d8d6eb7132cf admin admin] PATH_INFO: `/v3/projects/678f3ac94c414898ad065452966e70a5` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:36.987740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18bcc2e-b0fc-4875-ab9e-d8d6eb7132cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=678f3ac94c414898ad065452966e70a5)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:36.989682 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18bcc2e-b0fc-4875-ab9e-d8d6eb7132cf admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:37.039256 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e18bcc2e-b0fc-4875-ab9e-d8d6eb7132cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '678f3ac94c414898ad065452966e70a5', 'request_id': 'req-e18bcc2e-b0fc-4875-ab9e-d8d6eb7132cf'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:37.046641 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e18bcc2e-b0fc-4875-ab9e-d8d6eb7132cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '678f3ac94c414898ad065452966e70a5', 'request_id': 'req-e18bcc2e-b0fc-4875-ab9e-d8d6eb7132cf'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:37.052272 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e18bcc2e-b0fc-4875-ab9e-d8d6eb7132cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '678f3ac94c414898ad065452966e70a5', 'request_id': 'req-e18bcc2e-b0fc-4875-ab9e-d8d6eb7132cf'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:37.063755 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e18bcc2e-b0fc-4875-ab9e-d8d6eb7132cf admin admin] The token cache is being invalidate because project 678f3ac94c414898ad065452966e70a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:59:37.064047 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e18bcc2e-b0fc-4875-ab9e-d8d6eb7132cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:37.064693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e18bcc2e-b0fc-4875-ab9e-d8d6eb7132cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:37.066218 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1297/2600] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:59:36 2022] DELETE /identity/v3/projects/678f3ac94c414898ad065452966e70a5 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 22:59:37.070944 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf5fb0-af5b-4c46-93e8-2334aa8398a1 service neutron] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:37.080693 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf5fb0-af5b-4c46-93e8-2334aa8398a1 service neutron] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:37.096391 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:37.139618 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1f37af-64db-4382-8781-dc6e92e7fc15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:37.140251 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f37af-64db-4382-8781-dc6e92e7fc15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:37.140536 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f37af-64db-4382-8781-dc6e92e7fc15 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:37.140815 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f37af-64db-4382-8781-dc6e92e7fc15 admin admin] PATH_INFO: `/v3/users/379a47fda05842de905933d9e360af49` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:37.141957 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1f37af-64db-4382-8781-dc6e92e7fc15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=379a47fda05842de905933d9e360af49)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:37.143899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1f37af-64db-4382-8781-dc6e92e7fc15 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:37.195748 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-4a1f37af-64db-4382-8781-dc6e92e7fc15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '379a47fda05842de905933d9e360af49'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:37.205551 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-4a1f37af-64db-4382-8781-dc6e92e7fc15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '379a47fda05842de905933d9e360af49'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:37.214203 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-4a1f37af-64db-4382-8781-dc6e92e7fc15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '379a47fda05842de905933d9e360af49'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:37.214862 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-4a1f37af-64db-4382-8781-dc6e92e7fc15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '379a47fda05842de905933d9e360af49'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:37.215473 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-4a1f37af-64db-4382-8781-dc6e92e7fc15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '379a47fda05842de905933d9e360af49'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:37.233359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1304/2601] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:59:37 2022] DELETE /identity/v3/users/379a47fda05842de905933d9e360af49 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:37.509356 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18bcc2e-b0fc-4875-ab9e-d8d6eb7132cf admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:37.510515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-e18bcc2e-b0fc-4875-ab9e-d8d6eb7132cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:37.526048 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18bcc2e-b0fc-4875-ab9e-d8d6eb7132cf admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:37.585460 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f18fd0-d42d-4823-9926-44c3dd944da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:37.586099 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f18fd0-d42d-4823-9926-44c3dd944da7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:37.586383 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f18fd0-d42d-4823-9926-44c3dd944da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:37.586661 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f18fd0-d42d-4823-9926-44c3dd944da7 admin admin] PATH_INFO: `/v3/projects/22f59c9c1d91405dab44aaff4075609e` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:37.587836 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f18fd0-d42d-4823-9926-44c3dd944da7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22f59c9c1d91405dab44aaff4075609e)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:37.589801 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f18fd0-d42d-4823-9926-44c3dd944da7 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:37.635962 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f8f18fd0-d42d-4823-9926-44c3dd944da7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22f59c9c1d91405dab44aaff4075609e', 'request_id': 'req-f8f18fd0-d42d-4823-9926-44c3dd944da7'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:37.643218 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f8f18fd0-d42d-4823-9926-44c3dd944da7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22f59c9c1d91405dab44aaff4075609e', 'request_id': 'req-f8f18fd0-d42d-4823-9926-44c3dd944da7'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:37.649043 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f8f18fd0-d42d-4823-9926-44c3dd944da7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22f59c9c1d91405dab44aaff4075609e', 'request_id': 'req-f8f18fd0-d42d-4823-9926-44c3dd944da7'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:37.660023 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f8f18fd0-d42d-4823-9926-44c3dd944da7 admin admin] The token cache is being invalidate because project 22f59c9c1d91405dab44aaff4075609e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:59:37.660327 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f8f18fd0-d42d-4823-9926-44c3dd944da7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:37.660996 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f8f18fd0-d42d-4823-9926-44c3dd944da7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:37.662488 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1298/2602] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:59:37 2022] DELETE /identity/v3/projects/22f59c9c1d91405dab44aaff4075609e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:37.667895 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dbd9d7-7c75-42a9-bea9-69ecd826de1b None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:37.668180 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dbd9d7-7c75-42a9-bea9-69ecd826de1b None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:37.668455 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dbd9d7-7c75-42a9-bea9-69ecd826de1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:37.688442 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-63dbd9d7-7c75-42a9-bea9-69ecd826de1b None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:59:37.689668 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-63dbd9d7-7c75-42a9-bea9-69ecd826de1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:37.750483 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1305/2603] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:59:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:59:37.758999 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ca7dc-a413-45bc-a7a5-c886732773c6 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:37.759227 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ca7dc-a413-45bc-a7a5-c886732773c6 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:37.759508 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ca7dc-a413-45bc-a7a5-c886732773c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:37.768177 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:37.780444 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-d97ca7dc-a413-45bc-a7a5-c886732773c6 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:59:37.782072 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-d97ca7dc-a413-45bc-a7a5-c886732773c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:37.790892 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1299/2604] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:59:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:59:37.795167 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dbd9d7-7c75-42a9-bea9-69ecd826de1b None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:37.805219 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dbd9d7-7c75-42a9-bea9-69ecd826de1b None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:37.809951 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f425e4-490a-43ea-b687-3c38e716d1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:37.810575 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f425e4-490a-43ea-b687-3c38e716d1dc admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:37.810856 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f425e4-490a-43ea-b687-3c38e716d1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:37.811155 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f425e4-490a-43ea-b687-3c38e716d1dc admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:37.811807 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f425e4-490a-43ea-b687-3c38e716d1dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:59:37.812110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f425e4-490a-43ea-b687-3c38e716d1dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:37.813754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f425e4-490a-43ea-b687-3c38e716d1dc admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:37.825670 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1306/2605] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:59:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 22:59:37.830093 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ca7dc-a413-45bc-a7a5-c886732773c6 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:37.840013 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ca7dc-a413-45bc-a7a5-c886732773c6 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:37.844720 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3280f081-c4f2-43a8-8504-8a7a2ae1f7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:37.845349 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280f081-c4f2-43a8-8504-8a7a2ae1f7ae admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:37.845634 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280f081-c4f2-43a8-8504-8a7a2ae1f7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:37.845910 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280f081-c4f2-43a8-8504-8a7a2ae1f7ae admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:37.846540 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3280f081-c4f2-43a8-8504-8a7a2ae1f7ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:37.848449 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3280f081-c4f2-43a8-8504-8a7a2ae1f7ae admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:37.876554 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1300/2606] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:59:37 2022] POST /identity/v3/projects => generated 357 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:59:37.880866 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f425e4-490a-43ea-b687-3c38e716d1dc admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:37.891126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f425e4-490a-43ea-b687-3c38e716d1dc admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:37.916543 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:37.950286 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf1ab73-ea46-4dad-989a-6e7ac9d2ff60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:37.950896 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1ab73-ea46-4dad-989a-6e7ac9d2ff60 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:37.951177 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1ab73-ea46-4dad-989a-6e7ac9d2ff60 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:37.951449 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1ab73-ea46-4dad-989a-6e7ac9d2ff60 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:37.952072 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf1ab73-ea46-4dad-989a-6e7ac9d2ff60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:37.954036 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf1ab73-ea46-4dad-989a-6e7ac9d2ff60 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:37.975504 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1307/2607] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:59:37 2022] POST /identity/v3/users => generated 279 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:59:37.980569 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3280f081-c4f2-43a8-8504-8a7a2ae1f7ae admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:37.990690 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3280f081-c4f2-43a8-8504-8a7a2ae1f7ae admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:37.995485 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee1569-9558-4a23-9926-ae9ee27644b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:37.996078 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee1569-9558-4a23-9926-ae9ee27644b6 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:37.996351 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee1569-9558-4a23-9926-ae9ee27644b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:37.996617 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee1569-9558-4a23-9926-ae9ee27644b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:37.997744 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ee1569-9558-4a23-9926-ae9ee27644b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:37.999315 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ee1569-9558-4a23-9926-ae9ee27644b6 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:38.009245 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1301/2608] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:59:38.013101 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf1ab73-ea46-4dad-989a-6e7ac9d2ff60 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:38.023438 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf1ab73-ea46-4dad-989a-6e7ac9d2ff60 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:38.028175 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0ec92-7944-497f-b70a-8894225700a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:38.028770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0ec92-7944-497f-b70a-8894225700a4 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:38.029049 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0ec92-7944-497f-b70a-8894225700a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:38.029366 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0ec92-7944-497f-b70a-8894225700a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:38.030005 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce0ec92-7944-497f-b70a-8894225700a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:38.031597 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce0ec92-7944-497f-b70a-8894225700a4 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:38.042936 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1308/2609] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:59:38.046957 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee1569-9558-4a23-9926-ae9ee27644b6 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:38.057140 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee1569-9558-4a23-9926-ae9ee27644b6 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:38.061698 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2827235-4fb4-4579-a307-d5f7f22188a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:38.062338 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2827235-4fb4-4579-a307-d5f7f22188a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:38.062613 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2827235-4fb4-4579-a307-d5f7f22188a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:38.062883 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2827235-4fb4-4579-a307-d5f7f22188a1 admin admin] PATH_INFO: `/v3/projects/4cd51b2713ce47c6a2512404b50b05d9/users/9bf714050b604bfc879449596e0d6dc9/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:38.077461 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2827235-4fb4-4579-a307-d5f7f22188a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cd51b2713ce47c6a2512404b50b05d9, user_id=9bf714050b604bfc879449596e0d6dc9, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:38.080162 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2827235-4fb4-4579-a307-d5f7f22188a1 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:38.094717 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1302/2610] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:59:38 2022] PUT /identity/v3/projects/4cd51b2713ce47c6a2512404b50b05d9/users/9bf714050b604bfc879449596e0d6dc9/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:38.106576 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6aba29-a9aa-440e-b0da-26694c94454f None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:38.106868 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6aba29-a9aa-440e-b0da-26694c94454f None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:38.107144 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6aba29-a9aa-440e-b0da-26694c94454f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:38.135472 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-6a6aba29-a9aa-440e-b0da-26694c94454f None None] MFA Rules not processed for user `9bf714050b604bfc879449596e0d6dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:59:38.136698 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-6a6aba29-a9aa-440e-b0da-26694c94454f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:38.201842 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1309/2611] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:59:38 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:59:38.241084 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2827235-4fb4-4579-a307-d5f7f22188a1 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:38.245948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-f2827235-4fb4-4579-a307-d5f7f22188a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:38.304797 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2827235-4fb4-4579-a307-d5f7f22188a1 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:38.308906 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b53282f-42e9-42e2-b74d-c259cda1fe51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:38.309549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53282f-42e9-42e2-b74d-c259cda1fe51 service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:38.309833 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53282f-42e9-42e2-b74d-c259cda1fe51 service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:38.310108 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53282f-42e9-42e2-b74d-c259cda1fe51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:38.318453 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b53282f-42e9-42e2-b74d-c259cda1fe51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:38.320770 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b53282f-42e9-42e2-b74d-c259cda1fe51 service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:38.329641 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1303/2612] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:59:38 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:59:38.345518 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6aba29-a9aa-440e-b0da-26694c94454f None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:38.407553 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6aba29-a9aa-440e-b0da-26694c94454f None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:38.412092 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22981d16-17db-4efc-bd39-494d723f3083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:38.412721 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22981d16-17db-4efc-bd39-494d723f3083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:38.412999 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22981d16-17db-4efc-bd39-494d723f3083 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:38.414270 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22981d16-17db-4efc-bd39-494d723f3083 admin admin] PATH_INFO: `/v3/users/9bf714050b604bfc879449596e0d6dc9` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:38.414527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22981d16-17db-4efc-bd39-494d723f3083 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bf714050b604bfc879449596e0d6dc9)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:38.416466 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22981d16-17db-4efc-bd39-494d723f3083 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:38.455003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:38.465440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-22981d16-17db-4efc-bd39-494d723f3083 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bf714050b604bfc879449596e0d6dc9'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:38.472680 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-22981d16-17db-4efc-bd39-494d723f3083 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bf714050b604bfc879449596e0d6dc9'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:38.480205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-22981d16-17db-4efc-bd39-494d723f3083 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bf714050b604bfc879449596e0d6dc9'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:38.480856 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-22981d16-17db-4efc-bd39-494d723f3083 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bf714050b604bfc879449596e0d6dc9'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:38.481486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-22981d16-17db-4efc-bd39-494d723f3083 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bf714050b604bfc879449596e0d6dc9'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:38.498720 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1310/2613] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 22:59:38 2022] DELETE /identity/v3/users/9bf714050b604bfc879449596e0d6dc9 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:38.503265 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ecc0b0-4fb4-46d5-a713-73c5e01a208e None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:38.503551 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ecc0b0-4fb4-46d5-a713-73c5e01a208e None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:38.503824 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ecc0b0-4fb4-46d5-a713-73c5e01a208e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:38.523397 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-19ecc0b0-4fb4-46d5-a713-73c5e01a208e None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:59:38.524628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-19ecc0b0-4fb4-46d5-a713-73c5e01a208e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:38.593732 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1304/2614] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:59:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:59:38.604181 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22981d16-17db-4efc-bd39-494d723f3083 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:38.605684 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-22981d16-17db-4efc-bd39-494d723f3083 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:38.664974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22981d16-17db-4efc-bd39-494d723f3083 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:38.668650 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0732d79-7d62-4141-8303-fc166e4d7eaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:38.669254 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0732d79-7d62-4141-8303-fc166e4d7eaa service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:38.669568 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0732d79-7d62-4141-8303-fc166e4d7eaa service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:38.669842 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0732d79-7d62-4141-8303-fc166e4d7eaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:38.679245 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0732d79-7d62-4141-8303-fc166e4d7eaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:38.680817 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0732d79-7d62-4141-8303-fc166e4d7eaa service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:38.690211 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1311/2615] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:59:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:59:38.975155 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ecc0b0-4fb4-46d5-a713-73c5e01a208e None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:38.976839 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-19ecc0b0-4fb4-46d5-a713-73c5e01a208e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:38.990160 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ecc0b0-4fb4-46d5-a713-73c5e01a208e None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:38.995663 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20afe951-1573-4d9f-948e-38bb0a93cb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:38.996408 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20afe951-1573-4d9f-948e-38bb0a93cb64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:38.996756 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20afe951-1573-4d9f-948e-38bb0a93cb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:38.997031 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20afe951-1573-4d9f-948e-38bb0a93cb64 admin admin] PATH_INFO: `/v3/projects/4cd51b2713ce47c6a2512404b50b05d9` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:38.998629 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20afe951-1573-4d9f-948e-38bb0a93cb64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cd51b2713ce47c6a2512404b50b05d9)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:39.000935 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20afe951-1573-4d9f-948e-38bb0a93cb64 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:39.169887 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:39.177793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-20afe951-1573-4d9f-948e-38bb0a93cb64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cd51b2713ce47c6a2512404b50b05d9', 'request_id': 'req-20afe951-1573-4d9f-948e-38bb0a93cb64'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:39.185292 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-20afe951-1573-4d9f-948e-38bb0a93cb64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cd51b2713ce47c6a2512404b50b05d9', 'request_id': 'req-20afe951-1573-4d9f-948e-38bb0a93cb64'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:39.190812 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-20afe951-1573-4d9f-948e-38bb0a93cb64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cd51b2713ce47c6a2512404b50b05d9', 'request_id': 'req-20afe951-1573-4d9f-948e-38bb0a93cb64'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:39.202313 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-20afe951-1573-4d9f-948e-38bb0a93cb64 admin admin] The token cache is being invalidate because project 4cd51b2713ce47c6a2512404b50b05d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 22:59:39.202595 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-20afe951-1573-4d9f-948e-38bb0a93cb64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:39.203254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-20afe951-1573-4d9f-948e-38bb0a93cb64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 22:59:39.204731 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1305/2616] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 22:59:38 2022] DELETE /identity/v3/projects/4cd51b2713ce47c6a2512404b50b05d9 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:39.209797 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c87f9-8874-433a-bf52-828ee1dd7553 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:39.210645 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c87f9-8874-433a-bf52-828ee1dd7553 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:39.211525 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c87f9-8874-433a-bf52-828ee1dd7553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:39.235339 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-9b8c87f9-8874-433a-bf52-828ee1dd7553 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:59:39.236571 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-9b8c87f9-8874-433a-bf52-828ee1dd7553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:39.301360 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1312/2617] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:59:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:59:39.364910 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e23fd-ec39-4b07-93aa-2298d3bade00 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:39.365190 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e23fd-ec39-4b07-93aa-2298d3bade00 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:39.365487 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e23fd-ec39-4b07-93aa-2298d3bade00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:39.385005 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-be3e23fd-ec39-4b07-93aa-2298d3bade00 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:59:39.386238 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-be3e23fd-ec39-4b07-93aa-2298d3bade00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:39.393978 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1306/2618] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:59:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:59:39.398849 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c87f9-8874-433a-bf52-828ee1dd7553 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:39.408474 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c87f9-8874-433a-bf52-828ee1dd7553 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:39.412977 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8921155-d518-4a9e-96d4-f0fd1232f703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:39.414295 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8921155-d518-4a9e-96d4-f0fd1232f703 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:39.414574 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8921155-d518-4a9e-96d4-f0fd1232f703 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:39.414911 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8921155-d518-4a9e-96d4-f0fd1232f703 admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:39.415471 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8921155-d518-4a9e-96d4-f0fd1232f703 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 22:59:39.415770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8921155-d518-4a9e-96d4-f0fd1232f703 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:39.417500 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8921155-d518-4a9e-96d4-f0fd1232f703 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:39.429206 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1313/2619] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 22:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 22:59:39.433672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e23fd-ec39-4b07-93aa-2298d3bade00 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:39.443056 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e23fd-ec39-4b07-93aa-2298d3bade00 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:39.447998 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6979e-6883-4b74-a9fb-6cf0b04e3bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:39.448596 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6979e-6883-4b74-a9fb-6cf0b04e3bc1 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:39.448870 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6979e-6883-4b74-a9fb-6cf0b04e3bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:39.449134 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6979e-6883-4b74-a9fb-6cf0b04e3bc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:39.449788 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a6979e-6883-4b74-a9fb-6cf0b04e3bc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:39.451686 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a6979e-6883-4b74-a9fb-6cf0b04e3bc1 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:39.478624 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1307/2620] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 22:59:39 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 22:59:39.482818 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8921155-d518-4a9e-96d4-f0fd1232f703 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:39.492925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8921155-d518-4a9e-96d4-f0fd1232f703 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:39.534707 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:39.552381 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d5b65-0033-451a-966d-0640e6c5e6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:39.553015 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d5b65-0033-451a-966d-0640e6c5e6f9 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:39.553308 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d5b65-0033-451a-966d-0640e6c5e6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:39.553614 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d5b65-0033-451a-966d-0640e6c5e6f9 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:39.554500 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d5b65-0033-451a-966d-0640e6c5e6f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:39.556466 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d5b65-0033-451a-966d-0640e6c5e6f9 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:39.581163 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1314/2621] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 22:59:39 2022] POST /identity/v3/users => generated 285 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 22:59:39.587655 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6979e-6883-4b74-a9fb-6cf0b04e3bc1 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:39.605638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6979e-6883-4b74-a9fb-6cf0b04e3bc1 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:39.610648 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31661381-7b2d-4741-8496-d7f476201bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:39.611292 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31661381-7b2d-4741-8496-d7f476201bec admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:39.611564 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31661381-7b2d-4741-8496-d7f476201bec admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:39.611829 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31661381-7b2d-4741-8496-d7f476201bec admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:39.612459 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31661381-7b2d-4741-8496-d7f476201bec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:39.614088 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31661381-7b2d-4741-8496-d7f476201bec admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:39.624950 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1308/2622] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 22:59:39.629391 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d5b65-0033-451a-966d-0640e6c5e6f9 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:39.640268 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d5b65-0033-451a-966d-0640e6c5e6f9 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:39.644839 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4736aca-20f7-4ca0-9d77-df94bbd57fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:39.645467 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4736aca-20f7-4ca0-9d77-df94bbd57fd3 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:39.645746 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4736aca-20f7-4ca0-9d77-df94bbd57fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:39.646021 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4736aca-20f7-4ca0-9d77-df94bbd57fd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:39.646646 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4736aca-20f7-4ca0-9d77-df94bbd57fd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:39.648905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4736aca-20f7-4ca0-9d77-df94bbd57fd3 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:39.659788 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1315/2623] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 22:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 22:59:39.664501 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31661381-7b2d-4741-8496-d7f476201bec admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:39.674403 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31661381-7b2d-4741-8496-d7f476201bec admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:39.679002 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7074f7cb-1246-47ec-91e6-777de7d9a78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:39.679700 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7074f7cb-1246-47ec-91e6-777de7d9a78d admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:39.679922 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7074f7cb-1246-47ec-91e6-777de7d9a78d admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:39.680197 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7074f7cb-1246-47ec-91e6-777de7d9a78d admin admin] PATH_INFO: `/v3/projects/a220bb3708334387872ed7839dbba0d1/users/b29fd534f32744f7897be562580bd96e/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:39.695086 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7074f7cb-1246-47ec-91e6-777de7d9a78d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a220bb3708334387872ed7839dbba0d1, user_id=b29fd534f32744f7897be562580bd96e, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:39.698257 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7074f7cb-1246-47ec-91e6-777de7d9a78d admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:39.711719 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1309/2624] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 22:59:39 2022] PUT /identity/v3/projects/a220bb3708334387872ed7839dbba0d1/users/b29fd534f32744f7897be562580bd96e/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 22:59:39.721962 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e28702-cad7-4185-9944-1ad1ed819832 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:39.722509 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e28702-cad7-4185-9944-1ad1ed819832 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:39.722509 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e28702-cad7-4185-9944-1ad1ed819832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:39.751373 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-b4e28702-cad7-4185-9944-1ad1ed819832 None None] MFA Rules not processed for user `b29fd534f32744f7897be562580bd96e`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 22:59:39.752593 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-b4e28702-cad7-4185-9944-1ad1ed819832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:39.815730 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1316/2625] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 22:59:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 22:59:39.828293 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7074f7cb-1246-47ec-91e6-777de7d9a78d admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 22:59:39.829974 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-7074f7cb-1246-47ec-91e6-777de7d9a78d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 22:59:39.919131 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7074f7cb-1246-47ec-91e6-777de7d9a78d admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 22:59:39.923128 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e949a8-9ea2-462b-a030-0b930f40a7da service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc7524edf5d5422f91d92fd91be5ee1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 22:59:39.923858 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e949a8-9ea2-462b-a030-0b930f40a7da service glance] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 22:59:39.924139 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e949a8-9ea2-462b-a030-0b930f40a7da service glance] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 22:59:39.924411 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e949a8-9ea2-462b-a030-0b930f40a7da service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 22:59:39.933062 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e949a8-9ea2-462b-a030-0b930f40a7da service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 22:59:39.934760 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e949a8-9ea2-462b-a030-0b930f40a7da service glance] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 22:59:39.943614 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1310/2626] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 22:59:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 22:59:49.812759 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=46,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 22:59:49.939696 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=72,keystone:DELETE=4,keystone:INSERT=3 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:03.744808 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e28702-cad7-4185-9944-1ad1ed819832 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:03.746674 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-b4e28702-cad7-4185-9944-1ad1ed819832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:03.802502 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e28702-cad7-4185-9944-1ad1ed819832 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:03.806095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845bbb51-2ebe-4040-99b5-0653b12f6f7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:03.806697 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845bbb51-2ebe-4040-99b5-0653b12f6f7b service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:03.806974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845bbb51-2ebe-4040-99b5-0653b12f6f7b service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:03.807247 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845bbb51-2ebe-4040-99b5-0653b12f6f7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:03.808985 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-845bbb51-2ebe-4040-99b5-0653b12f6f7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:03.875867 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845bbb51-2ebe-4040-99b5-0653b12f6f7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:03.877373 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845bbb51-2ebe-4040-99b5-0653b12f6f7b service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:03.887530 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1317/2627] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:00:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:00:04.371302 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e949a8-9ea2-462b-a030-0b930f40a7da service glance] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:04.372759 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-47e949a8-9ea2-462b-a030-0b930f40a7da service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:04.385711 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e949a8-9ea2-462b-a030-0b930f40a7da service glance] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:04.390587 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d2f6d-2f43-464f-98ea-c574b5d75b22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:04.391285 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d2f6d-2f43-464f-98ea-c574b5d75b22 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:04.391651 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d2f6d-2f43-464f-98ea-c574b5d75b22 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:04.391970 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d2f6d-2f43-464f-98ea-c574b5d75b22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:04.393872 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-7c4d2f6d-2f43-464f-98ea-c574b5d75b22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:04.406979 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d2f6d-2f43-464f-98ea-c574b5d75b22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:04.408580 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d2f6d-2f43-464f-98ea-c574b5d75b22 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:04.419715 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1311/2628] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:00:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:00:07.287757 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845bbb51-2ebe-4040-99b5-0653b12f6f7b service neutron] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:07.290047 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-845bbb51-2ebe-4040-99b5-0653b12f6f7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:07.326425 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845bbb51-2ebe-4040-99b5-0653b12f6f7b service neutron] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:07.343103 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d91bb-b292-4ac7-86c2-be61250c5a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:07.344133 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d91bb-b292-4ac7-86c2-be61250c5a39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:07.344548 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d91bb-b292-4ac7-86c2-be61250c5a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:07.345009 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d91bb-b292-4ac7-86c2-be61250c5a39 admin admin] PATH_INFO: `/v3/users/12e6b0b71ecb4825aeab6735eb44e63e` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:07.348841 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d91bb-b292-4ac7-86c2-be61250c5a39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12e6b0b71ecb4825aeab6735eb44e63e)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:07.355110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d91bb-b292-4ac7-86c2-be61250c5a39 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:07.457201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-d55d91bb-b292-4ac7-86c2-be61250c5a39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12e6b0b71ecb4825aeab6735eb44e63e'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:07.471423 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-d55d91bb-b292-4ac7-86c2-be61250c5a39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12e6b0b71ecb4825aeab6735eb44e63e'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:07.486141 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-d55d91bb-b292-4ac7-86c2-be61250c5a39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12e6b0b71ecb4825aeab6735eb44e63e'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:07.489178 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-d55d91bb-b292-4ac7-86c2-be61250c5a39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12e6b0b71ecb4825aeab6735eb44e63e'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:07.491035 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-d55d91bb-b292-4ac7-86c2-be61250c5a39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12e6b0b71ecb4825aeab6735eb44e63e'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:07.521523 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1318/2629] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 23:00:07 2022] DELETE /identity/v3/users/12e6b0b71ecb4825aeab6735eb44e63e => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:07.910504 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d2f6d-2f43-464f-98ea-c574b5d75b22 service neutron] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:07.926595 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d2f6d-2f43-464f-98ea-c574b5d75b22 service neutron] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:08.053014 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435310cd-8362-4e81-8030-66177b0e5b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:08.053802 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435310cd-8362-4e81-8030-66177b0e5b79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:08.054153 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435310cd-8362-4e81-8030-66177b0e5b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:08.054517 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435310cd-8362-4e81-8030-66177b0e5b79 admin admin] PATH_INFO: `/v3/projects/a4e2567825f44d938dfbf7e043015367` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:08.056025 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435310cd-8362-4e81-8030-66177b0e5b79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4e2567825f44d938dfbf7e043015367)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:08.059254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435310cd-8362-4e81-8030-66177b0e5b79 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:08.160823 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-435310cd-8362-4e81-8030-66177b0e5b79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4e2567825f44d938dfbf7e043015367', 'request_id': 'req-435310cd-8362-4e81-8030-66177b0e5b79'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:08.179273 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-435310cd-8362-4e81-8030-66177b0e5b79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4e2567825f44d938dfbf7e043015367', 'request_id': 'req-435310cd-8362-4e81-8030-66177b0e5b79'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:08.193130 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-435310cd-8362-4e81-8030-66177b0e5b79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4e2567825f44d938dfbf7e043015367', 'request_id': 'req-435310cd-8362-4e81-8030-66177b0e5b79'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:08.210799 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-435310cd-8362-4e81-8030-66177b0e5b79 admin admin] The token cache is being invalidate because project a4e2567825f44d938dfbf7e043015367 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:00:08.211099 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-435310cd-8362-4e81-8030-66177b0e5b79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:08.211770 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-435310cd-8362-4e81-8030-66177b0e5b79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:08.213340 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1312/2630] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 23:00:07 2022] DELETE /identity/v3/projects/a4e2567825f44d938dfbf7e043015367 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:08.219084 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a81ab0e-6dee-42cd-9204-aee91334312c None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:08.219783 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a81ab0e-6dee-42cd-9204-aee91334312c None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:08.220133 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a81ab0e-6dee-42cd-9204-aee91334312c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:08.245779 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-0a81ab0e-6dee-42cd-9204-aee91334312c None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:08.247368 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-0a81ab0e-6dee-42cd-9204-aee91334312c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:08.274181 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:08.332949 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1319/2631] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:00:08.342394 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccb799-bfd1-44f9-a052-2d8f892066f1 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:08.342689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccb799-bfd1-44f9-a052-2d8f892066f1 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:08.343221 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccb799-bfd1-44f9-a052-2d8f892066f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:08.373009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-b2ccb799-bfd1-44f9-a052-2d8f892066f1 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:08.374604 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-b2ccb799-bfd1-44f9-a052-2d8f892066f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:08.384743 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1313/2632] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:00:08.389258 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ccb799-bfd1-44f9-a052-2d8f892066f1 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:08.392764 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a81ab0e-6dee-42cd-9204-aee91334312c None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:08.394391 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-0a81ab0e-6dee-42cd-9204-aee91334312c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:08.400662 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ccb799-bfd1-44f9-a052-2d8f892066f1 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:08.406388 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfbc20f-865a-4933-b21d-7d393bbc9295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:08.407330 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfbc20f-865a-4933-b21d-7d393bbc9295 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:08.408063 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfbc20f-865a-4933-b21d-7d393bbc9295 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:08.408515 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfbc20f-865a-4933-b21d-7d393bbc9295 admin admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:08.409400 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfbc20f-865a-4933-b21d-7d393bbc9295 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:00:08.410615 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfbc20f-865a-4933-b21d-7d393bbc9295 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:08.412808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfbc20f-865a-4933-b21d-7d393bbc9295 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:08.413181 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a81ab0e-6dee-42cd-9204-aee91334312c None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:08.423221 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20390d6b-7840-43ff-82b0-3bb6565c8733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:08.423814 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20390d6b-7840-43ff-82b0-3bb6565c8733 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:08.424100 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20390d6b-7840-43ff-82b0-3bb6565c8733 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:08.424378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20390d6b-7840-43ff-82b0-3bb6565c8733 admin admin] PATH_INFO: `/v3/users/da22fc66a1d14fa983463331de110258` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:08.425616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20390d6b-7840-43ff-82b0-3bb6565c8733 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da22fc66a1d14fa983463331de110258)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:08.427148 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1314/2633] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 23:00:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:00:08.427965 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20390d6b-7840-43ff-82b0-3bb6565c8733 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:08.432052 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfbc20f-865a-4933-b21d-7d393bbc9295 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:08.443174 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfbc20f-865a-4933-b21d-7d393bbc9295 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:08.448685 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aace9eb0-590f-4fad-8aa7-dcdc98840902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:08.449607 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace9eb0-590f-4fad-8aa7-dcdc98840902 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:08.449886 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace9eb0-590f-4fad-8aa7-dcdc98840902 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:08.450161 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace9eb0-590f-4fad-8aa7-dcdc98840902 admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:08.450796 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aace9eb0-590f-4fad-8aa7-dcdc98840902 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:08.452755 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aace9eb0-590f-4fad-8aa7-dcdc98840902 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:08.480682 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-20390d6b-7840-43ff-82b0-3bb6565c8733 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da22fc66a1d14fa983463331de110258'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:08.485600 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1315/2634] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 23:00:08 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:00:08.486646 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aace9eb0-590f-4fad-8aa7-dcdc98840902 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:08.488932 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-20390d6b-7840-43ff-82b0-3bb6565c8733 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da22fc66a1d14fa983463331de110258'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:08.497403 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-20390d6b-7840-43ff-82b0-3bb6565c8733 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da22fc66a1d14fa983463331de110258'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:08.498141 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-20390d6b-7840-43ff-82b0-3bb6565c8733 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da22fc66a1d14fa983463331de110258'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:08.498776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-20390d6b-7840-43ff-82b0-3bb6565c8733 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da22fc66a1d14fa983463331de110258'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:08.516615 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1320/2635] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 23:00:08 2022] DELETE /identity/v3/users/da22fc66a1d14fa983463331de110258 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:08.529716 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:08.556712 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aace9eb0-590f-4fad-8aa7-dcdc98840902 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:08.618505 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d272b120-a873-4bbb-b311-fc90b196990a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:08.619150 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272b120-a873-4bbb-b311-fc90b196990a admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:08.619479 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272b120-a873-4bbb-b311-fc90b196990a admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:08.620014 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272b120-a873-4bbb-b311-fc90b196990a admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:08.620708 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d272b120-a873-4bbb-b311-fc90b196990a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:08.622720 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d272b120-a873-4bbb-b311-fc90b196990a admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:08.643292 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1316/2636] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 23:00:08 2022] POST /identity/v3/users => generated 288 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:00:08.647514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20390d6b-7840-43ff-82b0-3bb6565c8733 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:08.648954 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-20390d6b-7840-43ff-82b0-3bb6565c8733 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:08.662993 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20390d6b-7840-43ff-82b0-3bb6565c8733 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:08.668365 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae61aa1-5472-4829-9f42-ffb135973199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:08.668981 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae61aa1-5472-4829-9f42-ffb135973199 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:08.669260 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae61aa1-5472-4829-9f42-ffb135973199 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:08.669566 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae61aa1-5472-4829-9f42-ffb135973199 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:08.670210 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae61aa1-5472-4829-9f42-ffb135973199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:08.671812 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae61aa1-5472-4829-9f42-ffb135973199 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:08.682788 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1321/2637] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:00:08.686955 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d272b120-a873-4bbb-b311-fc90b196990a admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:08.698805 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d272b120-a873-4bbb-b311-fc90b196990a admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:08.704179 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf9706-034b-4e5d-8169-66f06650d2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:08.704784 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf9706-034b-4e5d-8169-66f06650d2f0 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:08.705057 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf9706-034b-4e5d-8169-66f06650d2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:08.705355 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf9706-034b-4e5d-8169-66f06650d2f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:08.706016 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccf9706-034b-4e5d-8169-66f06650d2f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:08.707628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccf9706-034b-4e5d-8169-66f06650d2f0 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:08.711614 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae61aa1-5472-4829-9f42-ffb135973199 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:08.713561 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-8ae61aa1-5472-4829-9f42-ffb135973199 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:08.722895 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1317/2638] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:00:08.723703 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf9706-034b-4e5d-8169-66f06650d2f0 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:08.734711 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae61aa1-5472-4829-9f42-ffb135973199 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:08.735948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf9706-034b-4e5d-8169-66f06650d2f0 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:08.740593 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe75df86-6fc3-45ca-829c-3b53c2ae268b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:08.741155 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25688a59-7e3e-4565-8de2-5692f29c3c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:08.741425 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75df86-6fc3-45ca-829c-3b53c2ae268b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:08.741730 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75df86-6fc3-45ca-829c-3b53c2ae268b admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:08.741873 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25688a59-7e3e-4565-8de2-5692f29c3c7d admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:08.741975 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75df86-6fc3-45ca-829c-3b53c2ae268b admin admin] PATH_INFO: `/v3/projects/89c0b68704e04cdd84f27922e4da5e5d` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:08.742144 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25688a59-7e3e-4565-8de2-5692f29c3c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:08.742423 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25688a59-7e3e-4565-8de2-5692f29c3c7d admin admin] PATH_INFO: `/v3/projects/8ad2ca141f324bad88198144f6273f95/users/f667c2b27c1f4632968c874801c01bec/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:08.745507 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe75df86-6fc3-45ca-829c-3b53c2ae268b admin admin] RBAC: Authorizing `identity:delete_project(project_id=89c0b68704e04cdd84f27922e4da5e5d)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:08.746338 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe75df86-6fc3-45ca-829c-3b53c2ae268b admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:08.765003 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25688a59-7e3e-4565-8de2-5692f29c3c7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ad2ca141f324bad88198144f6273f95, user_id=f667c2b27c1f4632968c874801c01bec, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:08.767696 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25688a59-7e3e-4565-8de2-5692f29c3c7d admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:08.781741 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1318/2639] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 23:00:08 2022] PUT /identity/v3/projects/8ad2ca141f324bad88198144f6273f95/users/f667c2b27c1f4632968c874801c01bec/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:08.792544 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ff06b-df44-4088-becc-32120c194176 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:08.793101 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ff06b-df44-4088-becc-32120c194176 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:08.793101 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ff06b-df44-4088-becc-32120c194176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:08.797719 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-fe75df86-6fc3-45ca-829c-3b53c2ae268b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89c0b68704e04cdd84f27922e4da5e5d', 'request_id': 'req-fe75df86-6fc3-45ca-829c-3b53c2ae268b'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:08.804040 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-fe75df86-6fc3-45ca-829c-3b53c2ae268b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89c0b68704e04cdd84f27922e4da5e5d', 'request_id': 'req-fe75df86-6fc3-45ca-829c-3b53c2ae268b'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:08.812665 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-fe75df86-6fc3-45ca-829c-3b53c2ae268b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89c0b68704e04cdd84f27922e4da5e5d', 'request_id': 'req-fe75df86-6fc3-45ca-829c-3b53c2ae268b'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:08.825299 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-9f2ff06b-df44-4088-becc-32120c194176 None None] MFA Rules not processed for user `f667c2b27c1f4632968c874801c01bec`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:08.826793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-9f2ff06b-df44-4088-becc-32120c194176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:08.829578 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-fe75df86-6fc3-45ca-829c-3b53c2ae268b admin admin] The token cache is being invalidate because project 89c0b68704e04cdd84f27922e4da5e5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:00:08.829864 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-fe75df86-6fc3-45ca-829c-3b53c2ae268b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:08.830525 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-fe75df86-6fc3-45ca-829c-3b53c2ae268b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:08.835519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1322/2640] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 23:00:08 2022] DELETE /identity/v3/projects/89c0b68704e04cdd84f27922e4da5e5d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:00:08.837131 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb51f9-026c-4b99-946f-a9d2a91420b9 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:08.837540 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb51f9-026c-4b99-946f-a9d2a91420b9 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:08.837914 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb51f9-026c-4b99-946f-a9d2a91420b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:08.852498 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:08.860397 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-4ebb51f9-026c-4b99-946f-a9d2a91420b9 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:08.862204 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-4ebb51f9-026c-4b99-946f-a9d2a91420b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:08.912425 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1319/2641] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:00:08.924560 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ff06b-df44-4088-becc-32120c194176 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:08.926015 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-9f2ff06b-df44-4088-becc-32120c194176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:08.934776 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1323/2642] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:08.942549 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bbf060-7f6a-4271-b424-d51b05079a6c None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:08.942848 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bbf060-7f6a-4271-b424-d51b05079a6c None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:08.943422 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bbf060-7f6a-4271-b424-d51b05079a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:08.963586 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:08.969407 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-31bbf060-7f6a-4271-b424-d51b05079a6c None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:08.970633 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-31bbf060-7f6a-4271-b424-d51b05079a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:08.982993 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1324/2643] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:00:08.983961 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bbf060-7f6a-4271-b424-d51b05079a6c None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:08.993444 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bbf060-7f6a-4271-b424-d51b05079a6c None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:08.998904 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef4397-62bc-404c-8861-d79cd63cb6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:08.999610 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef4397-62bc-404c-8861-d79cd63cb6c4 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:08.999893 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef4397-62bc-404c-8861-d79cd63cb6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:09.000166 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef4397-62bc-404c-8861-d79cd63cb6c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:09.000812 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef4397-62bc-404c-8861-d79cd63cb6c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:00:09.001139 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef4397-62bc-404c-8861-d79cd63cb6c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:09.002788 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef4397-62bc-404c-8861-d79cd63cb6c4 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:09.019769 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1325/2644] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 23:00:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:00:09.024301 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef4397-62bc-404c-8861-d79cd63cb6c4 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:09.025896 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ff06b-df44-4088-becc-32120c194176 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:09.034776 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46becce-d095-4a67-945d-8f10a0f23bc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:09.035458 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46becce-d095-4a67-945d-8f10a0f23bc3 service nova] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:09.035733 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46becce-d095-4a67-945d-8f10a0f23bc3 service nova] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:09.036009 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46becce-d095-4a67-945d-8f10a0f23bc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:09.036130 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef4397-62bc-404c-8861-d79cd63cb6c4 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:09.042625 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd03db8a-c447-456a-9b3e-582d58f05be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:09.043304 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03db8a-c447-456a-9b3e-582d58f05be0 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:09.043689 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03db8a-c447-456a-9b3e-582d58f05be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:09.044025 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03db8a-c447-456a-9b3e-582d58f05be0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:09.044717 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd03db8a-c447-456a-9b3e-582d58f05be0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:09.047172 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd03db8a-c447-456a-9b3e-582d58f05be0 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:09.051380 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46becce-d095-4a67-945d-8f10a0f23bc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:09.053127 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46becce-d095-4a67-945d-8f10a0f23bc3 service nova] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:09.064341 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1320/2645] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:00:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:00:09.085351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1326/2646] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 23:00:09 2022] POST /identity/v3/projects => generated 367 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:00:09.089706 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46becce-d095-4a67-945d-8f10a0f23bc3 service nova] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:09.109554 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46becce-d095-4a67-945d-8f10a0f23bc3 service nova] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:09.179739 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7bad22-18af-45a4-8dcb-727eacb9d282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:09.180356 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bad22-18af-45a4-8dcb-727eacb9d282 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:09.180640 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bad22-18af-45a4-8dcb-727eacb9d282 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:09.180915 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bad22-18af-45a4-8dcb-727eacb9d282 admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:09.181617 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7bad22-18af-45a4-8dcb-727eacb9d282 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:09.185267 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7bad22-18af-45a4-8dcb-727eacb9d282 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:09.220003 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1321/2647] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 23:00:09 2022] POST /identity/v3/users => generated 284 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:00:09.221555 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd03db8a-c447-456a-9b3e-582d58f05be0 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:09.236259 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd03db8a-c447-456a-9b3e-582d58f05be0 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:09.244008 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9c16e-4fa1-4a2f-ac6f-0c8eb327fa6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:09.245748 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9c16e-4fa1-4a2f-ac6f-0c8eb327fa6d admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:09.246180 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9c16e-4fa1-4a2f-ac6f-0c8eb327fa6d admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:09.246594 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9c16e-4fa1-4a2f-ac6f-0c8eb327fa6d admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:09.247619 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f9c16e-4fa1-4a2f-ac6f-0c8eb327fa6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:09.251232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f9c16e-4fa1-4a2f-ac6f-0c8eb327fa6d admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:09.263760 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1327/2648] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:09 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:00:09.274265 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7bad22-18af-45a4-8dcb-727eacb9d282 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:09.284118 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7bad22-18af-45a4-8dcb-727eacb9d282 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:09.292184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c45ddda-cb9e-4392-bc66-a61112cff7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:09.292898 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c45ddda-cb9e-4392-bc66-a61112cff7e2 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:09.293226 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c45ddda-cb9e-4392-bc66-a61112cff7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:09.293470 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c45ddda-cb9e-4392-bc66-a61112cff7e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:09.294117 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c45ddda-cb9e-4392-bc66-a61112cff7e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:09.295725 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c45ddda-cb9e-4392-bc66-a61112cff7e2 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:09.310268 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1322/2649] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:09 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:00:09.314365 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9c16e-4fa1-4a2f-ac6f-0c8eb327fa6d admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:09.330115 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c45ddda-cb9e-4392-bc66-a61112cff7e2 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:09.331512 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9c16e-4fa1-4a2f-ac6f-0c8eb327fa6d admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:09.332166 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-8c45ddda-cb9e-4392-bc66-a61112cff7e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:09.336606 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd46e1-5550-4016-82ce-081a859b7ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:09.337388 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd46e1-5550-4016-82ce-081a859b7ffd admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:09.337781 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd46e1-5550-4016-82ce-081a859b7ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:09.338128 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd46e1-5550-4016-82ce-081a859b7ffd admin admin] PATH_INFO: `/v3/projects/4fa99c4c227e4b1289e9f5da1db3ff02/users/a65fea80f43a4f449001e09dcd632339/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:09.346435 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c45ddda-cb9e-4392-bc66-a61112cff7e2 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:09.352341 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a5564-8e9a-44f1-b1b7-367ee917a680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:09.353114 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a5564-8e9a-44f1-b1b7-367ee917a680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:09.353236 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a5564-8e9a-44f1-b1b7-367ee917a680 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:09.353571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a5564-8e9a-44f1-b1b7-367ee917a680 admin admin] PATH_INFO: `/v3/users/f667c2b27c1f4632968c874801c01bec` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:09.354439 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edd46e1-5550-4016-82ce-081a859b7ffd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fa99c4c227e4b1289e9f5da1db3ff02, user_id=a65fea80f43a4f449001e09dcd632339, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:09.354746 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5a5564-8e9a-44f1-b1b7-367ee917a680 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f667c2b27c1f4632968c874801c01bec)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:09.357373 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5a5564-8e9a-44f1-b1b7-367ee917a680 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:09.357847 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edd46e1-5550-4016-82ce-081a859b7ffd admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:09.373762 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1328/2650] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 23:00:09 2022] PUT /identity/v3/projects/4fa99c4c227e4b1289e9f5da1db3ff02/users/a65fea80f43a4f449001e09dcd632339/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:09.384181 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9cf4e-3f2f-489c-b419-8fcd143751f4 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:09.384468 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9cf4e-3f2f-489c-b419-8fcd143751f4 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:09.384745 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9cf4e-3f2f-489c-b419-8fcd143751f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:09.408800 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-ed5a5564-8e9a-44f1-b1b7-367ee917a680 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f667c2b27c1f4632968c874801c01bec'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:09.413412 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-83e9cf4e-3f2f-489c-b419-8fcd143751f4 None None] MFA Rules not processed for user `a65fea80f43a4f449001e09dcd632339`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:09.415084 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-83e9cf4e-3f2f-489c-b419-8fcd143751f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:09.416634 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-ed5a5564-8e9a-44f1-b1b7-367ee917a680 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f667c2b27c1f4632968c874801c01bec'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:09.433639 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-ed5a5564-8e9a-44f1-b1b7-367ee917a680 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f667c2b27c1f4632968c874801c01bec'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:09.434577 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-ed5a5564-8e9a-44f1-b1b7-367ee917a680 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f667c2b27c1f4632968c874801c01bec'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:09.435242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-ed5a5564-8e9a-44f1-b1b7-367ee917a680 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f667c2b27c1f4632968c874801c01bec'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:09.448377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1323/2651] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 23:00:09 2022] DELETE /identity/v3/users/f667c2b27c1f4632968c874801c01bec => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:09.453020 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6e2d1-0e50-47b5-93e6-d6065a4a21ac None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:09.453338 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6e2d1-0e50-47b5-93e6-d6065a4a21ac None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:09.453641 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6e2d1-0e50-47b5-93e6-d6065a4a21ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:09.461552 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:09.476428 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-b4f6e2d1-0e50-47b5-93e6-d6065a4a21ac None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:09.477678 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-b4f6e2d1-0e50-47b5-93e6-d6065a4a21ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:09.490024 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1329/2652] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:09.501012 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e9cf4e-3f2f-489c-b419-8fcd143751f4 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:09.502402 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-83e9cf4e-3f2f-489c-b419-8fcd143751f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:09.524202 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:09.540093 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1324/2653] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:00:09.550947 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f6e2d1-0e50-47b5-93e6-d6065a4a21ac None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:09.552322 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-b4f6e2d1-0e50-47b5-93e6-d6065a4a21ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:09.562093 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e9cf4e-3f2f-489c-b419-8fcd143751f4 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:09.622490 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f6e2d1-0e50-47b5-93e6-d6065a4a21ac None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:09.626453 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd6593-5864-46db-baef-e9ace32efc13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:09.627052 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd6593-5864-46db-baef-e9ace32efc13 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:09.627328 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd6593-5864-46db-baef-e9ace32efc13 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:09.627586 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd6593-5864-46db-baef-e9ace32efc13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:09.629642 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab86f44-d123-45dc-a87e-9fefe6891ce9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:09.630255 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab86f44-d123-45dc-a87e-9fefe6891ce9 service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:09.630532 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab86f44-d123-45dc-a87e-9fefe6891ce9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:09.630802 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab86f44-d123-45dc-a87e-9fefe6891ce9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:09.640721 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd6593-5864-46db-baef-e9ace32efc13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:09.643111 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd6593-5864-46db-baef-e9ace32efc13 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:09.655253 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1325/2654] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:00:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:00:09.693449 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab86f44-d123-45dc-a87e-9fefe6891ce9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:09.694992 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab86f44-d123-45dc-a87e-9fefe6891ce9 service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:09.705418 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1330/2655] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:00:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:00:09.946743 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd6593-5864-46db-baef-e9ace32efc13 service neutron] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:09.949954 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-3ddd6593-5864-46db-baef-e9ace32efc13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:09.964394 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd6593-5864-46db-baef-e9ace32efc13 service neutron] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:09.970528 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76faa90a-4eaf-4824-803b-f67dbe327e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:09.971175 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76faa90a-4eaf-4824-803b-f67dbe327e99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:09.971452 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76faa90a-4eaf-4824-803b-f67dbe327e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:09.971726 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76faa90a-4eaf-4824-803b-f67dbe327e99 admin admin] PATH_INFO: `/v3/projects/8ad2ca141f324bad88198144f6273f95` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:09.972888 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76faa90a-4eaf-4824-803b-f67dbe327e99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ad2ca141f324bad88198144f6273f95)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:09.974967 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76faa90a-4eaf-4824-803b-f67dbe327e99 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:10.041350 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-76faa90a-4eaf-4824-803b-f67dbe327e99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ad2ca141f324bad88198144f6273f95', 'request_id': 'req-76faa90a-4eaf-4824-803b-f67dbe327e99'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:10.049887 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-76faa90a-4eaf-4824-803b-f67dbe327e99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ad2ca141f324bad88198144f6273f95', 'request_id': 'req-76faa90a-4eaf-4824-803b-f67dbe327e99'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:10.056075 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-76faa90a-4eaf-4824-803b-f67dbe327e99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ad2ca141f324bad88198144f6273f95', 'request_id': 'req-76faa90a-4eaf-4824-803b-f67dbe327e99'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:10.068682 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-76faa90a-4eaf-4824-803b-f67dbe327e99 admin admin] The token cache is being invalidate because project 8ad2ca141f324bad88198144f6273f95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:00:10.068974 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-76faa90a-4eaf-4824-803b-f67dbe327e99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:10.069757 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-76faa90a-4eaf-4824-803b-f67dbe327e99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:10.071565 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1326/2656] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 23:00:09 2022] DELETE /identity/v3/projects/8ad2ca141f324bad88198144f6273f95 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:10.077684 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac35689-6f59-4bfe-946f-c7b7ced35de3 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:10.078253 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac35689-6f59-4bfe-946f-c7b7ced35de3 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:10.078691 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac35689-6f59-4bfe-946f-c7b7ced35de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:10.099210 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-aac35689-6f59-4bfe-946f-c7b7ced35de3 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:10.100393 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-aac35689-6f59-4bfe-946f-c7b7ced35de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:10.161339 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1331/2657] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:10.169050 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca89d9-57c1-4732-b2f9-1e0b19ae7391 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:10.169375 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca89d9-57c1-4732-b2f9-1e0b19ae7391 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:10.169707 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca89d9-57c1-4732-b2f9-1e0b19ae7391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:10.191025 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-faca89d9-57c1-4732-b2f9-1e0b19ae7391 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:10.192278 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-faca89d9-57c1-4732-b2f9-1e0b19ae7391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:10.200980 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1327/2658] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:10.205236 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac35689-6f59-4bfe-946f-c7b7ced35de3 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:10.215691 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac35689-6f59-4bfe-946f-c7b7ced35de3 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:10.220292 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07fac9b-d194-46d2-86a1-45145d31ade1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:10.220901 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07fac9b-d194-46d2-86a1-45145d31ade1 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:10.221186 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07fac9b-d194-46d2-86a1-45145d31ade1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:10.221496 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07fac9b-d194-46d2-86a1-45145d31ade1 admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:10.222142 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07fac9b-d194-46d2-86a1-45145d31ade1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:00:10.222448 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07fac9b-d194-46d2-86a1-45145d31ade1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:10.224150 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07fac9b-d194-46d2-86a1-45145d31ade1 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:10.236793 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1332/2659] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 23:00:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:00:10.241268 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca89d9-57c1-4732-b2f9-1e0b19ae7391 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:10.251666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca89d9-57c1-4732-b2f9-1e0b19ae7391 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:10.256354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b9b6ea-ee4e-41b2-a131-70fec83fbe0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:10.256963 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9b6ea-ee4e-41b2-a131-70fec83fbe0f admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:10.257243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9b6ea-ee4e-41b2-a131-70fec83fbe0f admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:10.257545 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9b6ea-ee4e-41b2-a131-70fec83fbe0f admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:10.258201 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b9b6ea-ee4e-41b2-a131-70fec83fbe0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:10.260119 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b9b6ea-ee4e-41b2-a131-70fec83fbe0f admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:10.284449 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:10.292616 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1328/2660] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 23:00:10 2022] POST /identity/v3/projects => generated 363 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:00:10.297675 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07fac9b-d194-46d2-86a1-45145d31ade1 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:10.309831 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07fac9b-d194-46d2-86a1-45145d31ade1 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:10.322767 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=100 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:10.367870 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b2583-ac87-4921-9604-966e17d65077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:10.368510 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b2583-ac87-4921-9604-966e17d65077 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:10.368743 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b2583-ac87-4921-9604-966e17d65077 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:10.369016 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b2583-ac87-4921-9604-966e17d65077 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:10.370581 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b2583-ac87-4921-9604-966e17d65077 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:10.372624 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b2583-ac87-4921-9604-966e17d65077 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:10.393240 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1333/2661] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 23:00:10 2022] POST /identity/v3/users => generated 282 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:00:10.397398 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b9b6ea-ee4e-41b2-a131-70fec83fbe0f admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:10.407952 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b9b6ea-ee4e-41b2-a131-70fec83fbe0f admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:10.412512 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaedf7fc-a6ee-4dba-ac4a-6e0e20878a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:10.413151 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaedf7fc-a6ee-4dba-ac4a-6e0e20878a75 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:10.413438 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaedf7fc-a6ee-4dba-ac4a-6e0e20878a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:10.413705 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaedf7fc-a6ee-4dba-ac4a-6e0e20878a75 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:10.414329 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaedf7fc-a6ee-4dba-ac4a-6e0e20878a75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:10.415954 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaedf7fc-a6ee-4dba-ac4a-6e0e20878a75 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:10.431966 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1329/2662] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:00:10.433966 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b2583-ac87-4921-9604-966e17d65077 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:10.444311 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b2583-ac87-4921-9604-966e17d65077 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:10.448915 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bac2f0-5eca-4e28-9157-206514990c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:10.449544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bac2f0-5eca-4e28-9157-206514990c53 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:10.449826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bac2f0-5eca-4e28-9157-206514990c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:10.450101 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bac2f0-5eca-4e28-9157-206514990c53 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:10.450739 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bac2f0-5eca-4e28-9157-206514990c53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:10.452340 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bac2f0-5eca-4e28-9157-206514990c53 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:10.462152 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1334/2663] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:00:10.466598 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaedf7fc-a6ee-4dba-ac4a-6e0e20878a75 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:10.476501 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaedf7fc-a6ee-4dba-ac4a-6e0e20878a75 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:10.482126 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f377d7cc-d334-46bc-9d5f-3ec4f3d1c37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:10.482758 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377d7cc-d334-46bc-9d5f-3ec4f3d1c37b admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:10.483038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377d7cc-d334-46bc-9d5f-3ec4f3d1c37b admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:10.483311 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377d7cc-d334-46bc-9d5f-3ec4f3d1c37b admin admin] PATH_INFO: `/v3/projects/61ab5f50e2a9446fb0658c73a0d380b8/users/e59bed577ceb4c5c89f85dda6d11023c/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:10.498872 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f377d7cc-d334-46bc-9d5f-3ec4f3d1c37b admin admin] RBAC: Authorizing `identity:create_grant(project_id=61ab5f50e2a9446fb0658c73a0d380b8, user_id=e59bed577ceb4c5c89f85dda6d11023c, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:10.501551 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f377d7cc-d334-46bc-9d5f-3ec4f3d1c37b admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:10.514702 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1330/2664] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 23:00:10 2022] PUT /identity/v3/projects/61ab5f50e2a9446fb0658c73a0d380b8/users/e59bed577ceb4c5c89f85dda6d11023c/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:10.526256 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7db60d-9f92-4478-bed7-1dab7316111e None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:10.526585 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7db60d-9f92-4478-bed7-1dab7316111e None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:10.526896 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7db60d-9f92-4478-bed7-1dab7316111e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:10.556157 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-6b7db60d-9f92-4478-bed7-1dab7316111e None None] MFA Rules not processed for user `e59bed577ceb4c5c89f85dda6d11023c`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:10.558035 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-6b7db60d-9f92-4478-bed7-1dab7316111e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:10.632714 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1335/2665] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:10.637382 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f377d7cc-d334-46bc-9d5f-3ec4f3d1c37b admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:10.704224 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f377d7cc-d334-46bc-9d5f-3ec4f3d1c37b admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:10.709017 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415f0e4-04d9-44d7-98bb-9e54129d1c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:10.709641 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415f0e4-04d9-44d7-98bb-9e54129d1c73 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:10.709926 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415f0e4-04d9-44d7-98bb-9e54129d1c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:10.710197 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415f0e4-04d9-44d7-98bb-9e54129d1c73 admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:10.710824 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c415f0e4-04d9-44d7-98bb-9e54129d1c73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:10.712733 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c415f0e4-04d9-44d7-98bb-9e54129d1c73 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:10.740809 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1331/2666] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 23:00:10 2022] POST /identity/v3/projects => generated 365 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:00:10.745272 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7db60d-9f92-4478-bed7-1dab7316111e None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:10.755936 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7db60d-9f92-4478-bed7-1dab7316111e None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:10.812717 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c844d-c61b-4dc3-9039-2ec9b9b8edfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:10.813352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c844d-c61b-4dc3-9039-2ec9b9b8edfd admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:10.813638 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c844d-c61b-4dc3-9039-2ec9b9b8edfd admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:10.813916 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c844d-c61b-4dc3-9039-2ec9b9b8edfd admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:10.814541 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200c844d-c61b-4dc3-9039-2ec9b9b8edfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:10.816459 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200c844d-c61b-4dc3-9039-2ec9b9b8edfd admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:10.836391 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1336/2667] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 23:00:10 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:00:10.840954 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415f0e4-04d9-44d7-98bb-9e54129d1c73 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:10.851329 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415f0e4-04d9-44d7-98bb-9e54129d1c73 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:10.856753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a14d36-848b-4045-aab6-5ea09c832bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:10.857390 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a14d36-848b-4045-aab6-5ea09c832bb8 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:10.857674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a14d36-848b-4045-aab6-5ea09c832bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:10.857948 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a14d36-848b-4045-aab6-5ea09c832bb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:10.858628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a14d36-848b-4045-aab6-5ea09c832bb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:10.860221 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a14d36-848b-4045-aab6-5ea09c832bb8 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:10.870876 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1332/2668] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:00:10.875205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c844d-c61b-4dc3-9039-2ec9b9b8edfd admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:10.885160 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c844d-c61b-4dc3-9039-2ec9b9b8edfd admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:10.889840 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9eb37c-ae13-41f9-81df-3b4e0f7472fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:10.890481 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9eb37c-ae13-41f9-81df-3b4e0f7472fd admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:10.890761 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9eb37c-ae13-41f9-81df-3b4e0f7472fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:10.891033 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9eb37c-ae13-41f9-81df-3b4e0f7472fd admin admin] PATH_INFO: `/v3/projects/b66ac1c57cf040ce98cea4e7be21c6b3/users/38970ab65372412e93c844ebc0f61579/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:10.903747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:10.906976 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9eb37c-ae13-41f9-81df-3b4e0f7472fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b66ac1c57cf040ce98cea4e7be21c6b3, user_id=38970ab65372412e93c844ebc0f61579, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:10.909980 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9eb37c-ae13-41f9-81df-3b4e0f7472fd admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:10.924714 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1337/2669] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 23:00:10 2022] PUT /identity/v3/projects/b66ac1c57cf040ce98cea4e7be21c6b3/users/38970ab65372412e93c844ebc0f61579/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:00:10.928771 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a14d36-848b-4045-aab6-5ea09c832bb8 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:10.939504 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a14d36-848b-4045-aab6-5ea09c832bb8 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:10.999740 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa79b76-b62f-4ec8-a55b-011fd4fbacd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:11.000453 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa79b76-b62f-4ec8-a55b-011fd4fbacd3 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:11.000565 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa79b76-b62f-4ec8-a55b-011fd4fbacd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:11.000785 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa79b76-b62f-4ec8-a55b-011fd4fbacd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:11.001452 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa79b76-b62f-4ec8-a55b-011fd4fbacd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:11.003010 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa79b76-b62f-4ec8-a55b-011fd4fbacd3 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:11.013174 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1333/2670] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:10 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:00:11.017419 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9eb37c-ae13-41f9-81df-3b4e0f7472fd admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:11.028211 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9eb37c-ae13-41f9-81df-3b4e0f7472fd admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:11.035261 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baef6e6-9651-457a-b356-884141248bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:11.035923 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baef6e6-9651-457a-b356-884141248bd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:11.036447 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baef6e6-9651-457a-b356-884141248bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:11.036740 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baef6e6-9651-457a-b356-884141248bd6 admin admin] PATH_INFO: `/v3/projects/b66ac1c57cf040ce98cea4e7be21c6b3/users/38970ab65372412e93c844ebc0f61579/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:11.039054 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baef6e6-9651-457a-b356-884141248bd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b66ac1c57cf040ce98cea4e7be21c6b3, user_id=38970ab65372412e93c844ebc0f61579, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:11.041900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baef6e6-9651-457a-b356-884141248bd6 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:11.052461 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1338/2671] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 23:00:11 2022] PUT /identity/v3/projects/b66ac1c57cf040ce98cea4e7be21c6b3/users/38970ab65372412e93c844ebc0f61579/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:11.062990 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dcca96-6084-4eb8-a157-ec1fc192d0a9 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:11.063277 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dcca96-6084-4eb8-a157-ec1fc192d0a9 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:11.063587 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dcca96-6084-4eb8-a157-ec1fc192d0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:11.092447 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-01dcca96-6084-4eb8-a157-ec1fc192d0a9 None None] MFA Rules not processed for user `38970ab65372412e93c844ebc0f61579`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:11.093692 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-01dcca96-6084-4eb8-a157-ec1fc192d0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:11.116956 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:11.165353 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1334/2672] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:11 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:00:11.177078 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baef6e6-9651-457a-b356-884141248bd6 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:11.178861 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-2baef6e6-9651-457a-b356-884141248bd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:11.236834 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baef6e6-9651-457a-b356-884141248bd6 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:11.240550 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fbd68-22ce-4fee-9cb8-0683e45f15ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:11.241158 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fbd68-22ce-4fee-9cb8-0683e45f15ba service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:11.241471 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fbd68-22ce-4fee-9cb8-0683e45f15ba service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:11.241748 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fbd68-22ce-4fee-9cb8-0683e45f15ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:11.250848 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649fbd68-22ce-4fee-9cb8-0683e45f15ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:11.252217 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649fbd68-22ce-4fee-9cb8-0683e45f15ba service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:11.262054 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1339/2673] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:00:11 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:00:11.677910 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dcca96-6084-4eb8-a157-ec1fc192d0a9 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:11.742372 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dcca96-6084-4eb8-a157-ec1fc192d0a9 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:11.747350 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859ca607-9610-43db-af38-b7bc449be094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:11.747973 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ca607-9610-43db-af38-b7bc449be094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:11.748251 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ca607-9610-43db-af38-b7bc449be094 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:11.748522 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ca607-9610-43db-af38-b7bc449be094 admin admin] PATH_INFO: `/v3/users/e59bed577ceb4c5c89f85dda6d11023c` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:11.749691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859ca607-9610-43db-af38-b7bc449be094 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e59bed577ceb4c5c89f85dda6d11023c)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:11.751598 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859ca607-9610-43db-af38-b7bc449be094 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:11.797665 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-859ca607-9610-43db-af38-b7bc449be094 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e59bed577ceb4c5c89f85dda6d11023c'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:11.804354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-859ca607-9610-43db-af38-b7bc449be094 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e59bed577ceb4c5c89f85dda6d11023c'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:11.816878 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-859ca607-9610-43db-af38-b7bc449be094 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e59bed577ceb4c5c89f85dda6d11023c'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:11.817582 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-859ca607-9610-43db-af38-b7bc449be094 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e59bed577ceb4c5c89f85dda6d11023c'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:11.818199 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-859ca607-9610-43db-af38-b7bc449be094 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e59bed577ceb4c5c89f85dda6d11023c'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:11.830600 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1335/2674] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 23:00:11 2022] DELETE /identity/v3/users/e59bed577ceb4c5c89f85dda6d11023c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:11.835166 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12413121-ced9-4677-b539-df547d4a8fb1 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:11.835649 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12413121-ced9-4677-b539-df547d4a8fb1 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:11.835771 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12413121-ced9-4677-b539-df547d4a8fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:11.854670 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-12413121-ced9-4677-b539-df547d4a8fb1 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:11.856074 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-12413121-ced9-4677-b539-df547d4a8fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:11.917507 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1340/2675] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:11.927429 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859ca607-9610-43db-af38-b7bc449be094 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:11.928845 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-859ca607-9610-43db-af38-b7bc449be094 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:11.972074 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:11.987461 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859ca607-9610-43db-af38-b7bc449be094 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:11.991134 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e6628-9152-4d24-b047-8afaa8c21b56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:11.991713 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e6628-9152-4d24-b047-8afaa8c21b56 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:11.991989 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e6628-9152-4d24-b047-8afaa8c21b56 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:11.992257 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e6628-9152-4d24-b047-8afaa8c21b56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:12.002154 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351e6628-9152-4d24-b047-8afaa8c21b56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:12.003576 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351e6628-9152-4d24-b047-8afaa8c21b56 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:12.012988 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1336/2676] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:00:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:00:12.298208 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12413121-ced9-4677-b539-df547d4a8fb1 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:12.299720 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-12413121-ced9-4677-b539-df547d4a8fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:12.312190 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12413121-ced9-4677-b539-df547d4a8fb1 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:12.317763 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3eefbd-24c7-418f-96aa-1cafc516f3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:12.318625 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3eefbd-24c7-418f-96aa-1cafc516f3c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:12.319022 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3eefbd-24c7-418f-96aa-1cafc516f3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:12.319405 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3eefbd-24c7-418f-96aa-1cafc516f3c7 admin admin] PATH_INFO: `/v3/projects/61ab5f50e2a9446fb0658c73a0d380b8` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:12.320639 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3eefbd-24c7-418f-96aa-1cafc516f3c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61ab5f50e2a9446fb0658c73a0d380b8)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:12.322723 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3eefbd-24c7-418f-96aa-1cafc516f3c7 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:12.371339 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-bf3eefbd-24c7-418f-96aa-1cafc516f3c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61ab5f50e2a9446fb0658c73a0d380b8', 'request_id': 'req-bf3eefbd-24c7-418f-96aa-1cafc516f3c7'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:12.376625 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-bf3eefbd-24c7-418f-96aa-1cafc516f3c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61ab5f50e2a9446fb0658c73a0d380b8', 'request_id': 'req-bf3eefbd-24c7-418f-96aa-1cafc516f3c7'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:12.384464 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-bf3eefbd-24c7-418f-96aa-1cafc516f3c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61ab5f50e2a9446fb0658c73a0d380b8', 'request_id': 'req-bf3eefbd-24c7-418f-96aa-1cafc516f3c7'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:12.395513 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-bf3eefbd-24c7-418f-96aa-1cafc516f3c7 admin admin] The token cache is being invalidate because project 61ab5f50e2a9446fb0658c73a0d380b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:00:12.395793 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-bf3eefbd-24c7-418f-96aa-1cafc516f3c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:12.396414 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-bf3eefbd-24c7-418f-96aa-1cafc516f3c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:12.397921 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1341/2677] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 23:00:12 2022] DELETE /identity/v3/projects/61ab5f50e2a9446fb0658c73a0d380b8 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:12.402411 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e6628-9152-4d24-b047-8afaa8c21b56 service neutron] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:12.411856 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e6628-9152-4d24-b047-8afaa8c21b56 service neutron] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:12.469232 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3add93-7d8f-499b-8cef-a67301ad42a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:12.469892 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3add93-7d8f-499b-8cef-a67301ad42a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:12.470174 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3add93-7d8f-499b-8cef-a67301ad42a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:12.470450 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3add93-7d8f-499b-8cef-a67301ad42a2 admin admin] PATH_INFO: `/v3/users/38970ab65372412e93c844ebc0f61579` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:12.471594 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3add93-7d8f-499b-8cef-a67301ad42a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38970ab65372412e93c844ebc0f61579)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:12.473558 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3add93-7d8f-499b-8cef-a67301ad42a2 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:12.519619 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-dd3add93-7d8f-499b-8cef-a67301ad42a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38970ab65372412e93c844ebc0f61579'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:12.527381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-dd3add93-7d8f-499b-8cef-a67301ad42a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38970ab65372412e93c844ebc0f61579'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:12.538852 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-dd3add93-7d8f-499b-8cef-a67301ad42a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38970ab65372412e93c844ebc0f61579'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:12.539566 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-dd3add93-7d8f-499b-8cef-a67301ad42a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38970ab65372412e93c844ebc0f61579'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:12.540265 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-dd3add93-7d8f-499b-8cef-a67301ad42a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38970ab65372412e93c844ebc0f61579'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:12.553026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1337/2678] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 23:00:12 2022] DELETE /identity/v3/users/38970ab65372412e93c844ebc0f61579 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:12.823857 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3eefbd-24c7-418f-96aa-1cafc516f3c7 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:12.825547 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-bf3eefbd-24c7-418f-96aa-1cafc516f3c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:12.840044 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3eefbd-24c7-418f-96aa-1cafc516f3c7 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:12.840679 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:12.903468 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83d743-0391-441a-95c0-631a0e987663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:12.904029 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83d743-0391-441a-95c0-631a0e987663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:12.904340 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83d743-0391-441a-95c0-631a0e987663 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:12.904618 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83d743-0391-441a-95c0-631a0e987663 admin admin] PATH_INFO: `/v3/projects/b66ac1c57cf040ce98cea4e7be21c6b3` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:12.905782 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b83d743-0391-441a-95c0-631a0e987663 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b66ac1c57cf040ce98cea4e7be21c6b3)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:12.907812 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b83d743-0391-441a-95c0-631a0e987663 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:12.954144 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-7b83d743-0391-441a-95c0-631a0e987663 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b66ac1c57cf040ce98cea4e7be21c6b3', 'request_id': 'req-7b83d743-0391-441a-95c0-631a0e987663'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:12.959843 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-7b83d743-0391-441a-95c0-631a0e987663 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b66ac1c57cf040ce98cea4e7be21c6b3', 'request_id': 'req-7b83d743-0391-441a-95c0-631a0e987663'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:12.967558 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-7b83d743-0391-441a-95c0-631a0e987663 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b66ac1c57cf040ce98cea4e7be21c6b3', 'request_id': 'req-7b83d743-0391-441a-95c0-631a0e987663'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:12.978447 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-7b83d743-0391-441a-95c0-631a0e987663 admin admin] The token cache is being invalidate because project b66ac1c57cf040ce98cea4e7be21c6b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:00:12.978730 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-7b83d743-0391-441a-95c0-631a0e987663 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:12.979970 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-7b83d743-0391-441a-95c0-631a0e987663 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:12.981511 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1342/2679] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 23:00:12 2022] DELETE /identity/v3/projects/b66ac1c57cf040ce98cea4e7be21c6b3 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:12.986293 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17f6c4-ae08-43ca-9c04-ed239c177ac4 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:12.986582 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17f6c4-ae08-43ca-9c04-ed239c177ac4 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:12.986859 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17f6c4-ae08-43ca-9c04-ed239c177ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:13.007381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-da17f6c4-ae08-43ca-9c04-ed239c177ac4 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:13.008588 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-da17f6c4-ae08-43ca-9c04-ed239c177ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:13.069349 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1338/2680] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:13.077329 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687147b6-c7de-445f-b24b-a939cdb07434 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:13.077619 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687147b6-c7de-445f-b24b-a939cdb07434 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:13.077895 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687147b6-c7de-445f-b24b-a939cdb07434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:13.097730 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-687147b6-c7de-445f-b24b-a939cdb07434 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:13.098945 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-687147b6-c7de-445f-b24b-a939cdb07434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:13.107609 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1343/2681] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:13.111761 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da17f6c4-ae08-43ca-9c04-ed239c177ac4 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:13.116657 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:13.123838 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da17f6c4-ae08-43ca-9c04-ed239c177ac4 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:13.129472 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21521ce6-d1d9-475f-a926-833c56bdf7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:13.130103 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21521ce6-d1d9-475f-a926-833c56bdf7d9 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:13.130385 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21521ce6-d1d9-475f-a926-833c56bdf7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:13.130657 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21521ce6-d1d9-475f-a926-833c56bdf7d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:13.131293 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21521ce6-d1d9-475f-a926-833c56bdf7d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:00:13.131599 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21521ce6-d1d9-475f-a926-833c56bdf7d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:13.133180 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21521ce6-d1d9-475f-a926-833c56bdf7d9 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:13.145319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1339/2682] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 23:00:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 23:00:13.149582 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687147b6-c7de-445f-b24b-a939cdb07434 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:13.160010 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687147b6-c7de-445f-b24b-a939cdb07434 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:13.168109 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc3f2c2-603b-4e69-a34f-3d4c87a4f9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:13.168781 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3f2c2-603b-4e69-a34f-3d4c87a4f9ee admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:13.169062 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3f2c2-603b-4e69-a34f-3d4c87a4f9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:13.169363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3f2c2-603b-4e69-a34f-3d4c87a4f9ee admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:13.169992 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc3f2c2-603b-4e69-a34f-3d4c87a4f9ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:13.172023 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc3f2c2-603b-4e69-a34f-3d4c87a4f9ee admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:13.208192 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1344/2683] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 23:00:13 2022] POST /identity/v3/projects => generated 369 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:00:13.212305 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21521ce6-d1d9-475f-a926-833c56bdf7d9 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:13.223231 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21521ce6-d1d9-475f-a926-833c56bdf7d9 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:13.279753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc43ab-4b8c-44e8-a9b1-659e2844e787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:13.280424 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc43ab-4b8c-44e8-a9b1-659e2844e787 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:13.280699 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc43ab-4b8c-44e8-a9b1-659e2844e787 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:13.280966 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc43ab-4b8c-44e8-a9b1-659e2844e787 admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:13.281620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc43ab-4b8c-44e8-a9b1-659e2844e787 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:13.283520 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc43ab-4b8c-44e8-a9b1-659e2844e787 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:13.303877 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1340/2684] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 23:00:13 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:00:13.308608 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc3f2c2-603b-4e69-a34f-3d4c87a4f9ee admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:13.319058 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc3f2c2-603b-4e69-a34f-3d4c87a4f9ee admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:13.324586 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ad304-2b91-4aca-a7c2-c745ed7eca37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:13.325180 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ad304-2b91-4aca-a7c2-c745ed7eca37 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:13.325493 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ad304-2b91-4aca-a7c2-c745ed7eca37 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:13.325827 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ad304-2b91-4aca-a7c2-c745ed7eca37 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:13.326467 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73ad304-2b91-4aca-a7c2-c745ed7eca37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:13.328057 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73ad304-2b91-4aca-a7c2-c745ed7eca37 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:13.339382 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1345/2685] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:00:13.343319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc43ab-4b8c-44e8-a9b1-659e2844e787 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:13.353205 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc43ab-4b8c-44e8-a9b1-659e2844e787 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:13.358044 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87236b45-1ce9-4c56-94d1-9e40e299fc32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:13.358637 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87236b45-1ce9-4c56-94d1-9e40e299fc32 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:13.358913 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87236b45-1ce9-4c56-94d1-9e40e299fc32 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:13.359195 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87236b45-1ce9-4c56-94d1-9e40e299fc32 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:13.359804 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87236b45-1ce9-4c56-94d1-9e40e299fc32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:13.361408 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87236b45-1ce9-4c56-94d1-9e40e299fc32 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:13.372410 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1341/2686] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:00:13.376666 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ad304-2b91-4aca-a7c2-c745ed7eca37 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:13.388371 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ad304-2b91-4aca-a7c2-c745ed7eca37 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:13.393081 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080826cd-ad49-4061-8857-c12ff5ee3e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:13.393765 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080826cd-ad49-4061-8857-c12ff5ee3e79 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:13.394070 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080826cd-ad49-4061-8857-c12ff5ee3e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:13.394345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080826cd-ad49-4061-8857-c12ff5ee3e79 admin admin] PATH_INFO: `/v3/projects/8edcd6094aa84b27a37b03d4c6aa3bf3/users/eeb435c645d3464c990a2427e22d922f/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:13.409442 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080826cd-ad49-4061-8857-c12ff5ee3e79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8edcd6094aa84b27a37b03d4c6aa3bf3, user_id=eeb435c645d3464c990a2427e22d922f, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:13.412018 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080826cd-ad49-4061-8857-c12ff5ee3e79 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:13.425778 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1346/2687] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 23:00:13 2022] PUT /identity/v3/projects/8edcd6094aa84b27a37b03d4c6aa3bf3/users/eeb435c645d3464c990a2427e22d922f/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:00:13.437400 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45401fab-1d18-47dc-9887-9593fe6f0307 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:13.437687 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45401fab-1d18-47dc-9887-9593fe6f0307 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:13.437993 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45401fab-1d18-47dc-9887-9593fe6f0307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:13.466113 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-45401fab-1d18-47dc-9887-9593fe6f0307 None None] MFA Rules not processed for user `eeb435c645d3464c990a2427e22d922f`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:13.467320 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-45401fab-1d18-47dc-9887-9593fe6f0307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:13.530759 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1342/2688] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:13.535865 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080826cd-ad49-4061-8857-c12ff5ee3e79 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:13.574225 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:13.603884 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080826cd-ad49-4061-8857-c12ff5ee3e79 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:13.609738 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7087f54b-5cb2-4262-b1f0-1d759784933f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:13.610352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087f54b-5cb2-4262-b1f0-1d759784933f admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:13.610656 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087f54b-5cb2-4262-b1f0-1d759784933f admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:13.610929 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087f54b-5cb2-4262-b1f0-1d759784933f admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:13.611553 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7087f54b-5cb2-4262-b1f0-1d759784933f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:13.613490 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7087f54b-5cb2-4262-b1f0-1d759784933f admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:13.644559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1347/2689] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 23:00:13 2022] POST /identity/v3/projects => generated 367 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:00:13.645724 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45401fab-1d18-47dc-9887-9593fe6f0307 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:13.656596 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45401fab-1d18-47dc-9887-9593fe6f0307 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:13.706638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:13.713362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22180edd-67e3-4bfa-b608-dd5fbd5ea57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:13.714202 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22180edd-67e3-4bfa-b608-dd5fbd5ea57e admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:13.714741 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22180edd-67e3-4bfa-b608-dd5fbd5ea57e admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:13.714741 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22180edd-67e3-4bfa-b608-dd5fbd5ea57e admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:13.715417 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22180edd-67e3-4bfa-b608-dd5fbd5ea57e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:13.717644 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22180edd-67e3-4bfa-b608-dd5fbd5ea57e admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:13.738499 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1343/2690] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 23:00:13 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:00:13.742479 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7087f54b-5cb2-4262-b1f0-1d759784933f admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:13.753788 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7087f54b-5cb2-4262-b1f0-1d759784933f admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:13.758477 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc5eb6-6aa2-46aa-8f78-b4eeb54e0230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:13.759074 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc5eb6-6aa2-46aa-8f78-b4eeb54e0230 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:13.759357 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc5eb6-6aa2-46aa-8f78-b4eeb54e0230 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:13.759635 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc5eb6-6aa2-46aa-8f78-b4eeb54e0230 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:13.760914 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc5eb6-6aa2-46aa-8f78-b4eeb54e0230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:13.762554 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc5eb6-6aa2-46aa-8f78-b4eeb54e0230 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:13.772938 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1348/2691] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:00:13.777004 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22180edd-67e3-4bfa-b608-dd5fbd5ea57e admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:13.786943 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22180edd-67e3-4bfa-b608-dd5fbd5ea57e admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:13.791617 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a80cd6-2c3b-46bd-9e4b-b77841a98015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:13.792257 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a80cd6-2c3b-46bd-9e4b-b77841a98015 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:13.792536 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a80cd6-2c3b-46bd-9e4b-b77841a98015 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:13.792809 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a80cd6-2c3b-46bd-9e4b-b77841a98015 admin admin] PATH_INFO: `/v3/projects/585014d761af44dbb8ff1697288f4e3f/users/72a9f5b3810d4f6fb40655084aee1e5a/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:13.808281 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a80cd6-2c3b-46bd-9e4b-b77841a98015 admin admin] RBAC: Authorizing `identity:create_grant(project_id=585014d761af44dbb8ff1697288f4e3f, user_id=72a9f5b3810d4f6fb40655084aee1e5a, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:13.810971 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a80cd6-2c3b-46bd-9e4b-b77841a98015 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:13.824546 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1344/2692] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 23:00:13 2022] PUT /identity/v3/projects/585014d761af44dbb8ff1697288f4e3f/users/72a9f5b3810d4f6fb40655084aee1e5a/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:00:13.829057 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc5eb6-6aa2-46aa-8f78-b4eeb54e0230 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:13.839979 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc5eb6-6aa2-46aa-8f78-b4eeb54e0230 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:13.900358 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5751df2-e5aa-4207-a84b-36605a5caac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:13.901208 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5751df2-e5aa-4207-a84b-36605a5caac2 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:13.901456 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5751df2-e5aa-4207-a84b-36605a5caac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:13.901671 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5751df2-e5aa-4207-a84b-36605a5caac2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:13.902309 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5751df2-e5aa-4207-a84b-36605a5caac2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:13.903935 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5751df2-e5aa-4207-a84b-36605a5caac2 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:13.913650 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1349/2693] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:13 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:00:13.918106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a80cd6-2c3b-46bd-9e4b-b77841a98015 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:13.928351 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a80cd6-2c3b-46bd-9e4b-b77841a98015 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:13.933704 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463969d4-9bad-4427-805e-73260490a919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:13.934350 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463969d4-9bad-4427-805e-73260490a919 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:13.934644 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463969d4-9bad-4427-805e-73260490a919 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:13.934905 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463969d4-9bad-4427-805e-73260490a919 admin admin] PATH_INFO: `/v3/projects/585014d761af44dbb8ff1697288f4e3f/users/72a9f5b3810d4f6fb40655084aee1e5a/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:13.936831 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463969d4-9bad-4427-805e-73260490a919 admin admin] RBAC: Authorizing `identity:create_grant(project_id=585014d761af44dbb8ff1697288f4e3f, user_id=72a9f5b3810d4f6fb40655084aee1e5a, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:13.939527 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463969d4-9bad-4427-805e-73260490a919 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:13.951002 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1345/2694] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 23:00:13 2022] PUT /identity/v3/projects/585014d761af44dbb8ff1697288f4e3f/users/72a9f5b3810d4f6fb40655084aee1e5a/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:13.961186 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb34f9-67e5-43d0-9dbb-e67d8034e196 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:13.961446 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb34f9-67e5-43d0-9dbb-e67d8034e196 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:13.961732 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb34f9-67e5-43d0-9dbb-e67d8034e196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:13.989205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-d3cb34f9-67e5-43d0-9dbb-e67d8034e196 None None] MFA Rules not processed for user `72a9f5b3810d4f6fb40655084aee1e5a`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:13.990446 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-d3cb34f9-67e5-43d0-9dbb-e67d8034e196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:14.063452 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1350/2695] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:13 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:14.074192 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463969d4-9bad-4427-805e-73260490a919 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:14.075921 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-463969d4-9bad-4427-805e-73260490a919 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:14.133174 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463969d4-9bad-4427-805e-73260490a919 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:14.136917 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301fc2d3-a8f4-4dab-a06f-18ee9190fd85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:14.137544 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fc2d3-a8f4-4dab-a06f-18ee9190fd85 service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:14.137822 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fc2d3-a8f4-4dab-a06f-18ee9190fd85 service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:14.138093 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fc2d3-a8f4-4dab-a06f-18ee9190fd85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:14.147020 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301fc2d3-a8f4-4dab-a06f-18ee9190fd85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:14.148441 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301fc2d3-a8f4-4dab-a06f-18ee9190fd85 service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:14.157802 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1346/2696] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:00:14 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:00:15.675215 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cb34f9-67e5-43d0-9dbb-e67d8034e196 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:15.707171 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:15.743121 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cb34f9-67e5-43d0-9dbb-e67d8034e196 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:15.750110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a1439-fbba-4303-b643-b2da807e1ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:15.750870 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a1439-fbba-4303-b643-b2da807e1ee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:15.751090 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a1439-fbba-4303-b643-b2da807e1ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:15.751363 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a1439-fbba-4303-b643-b2da807e1ee1 admin admin] PATH_INFO: `/v3/users/eeb435c645d3464c990a2427e22d922f` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:15.752533 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a1439-fbba-4303-b643-b2da807e1ee1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eeb435c645d3464c990a2427e22d922f)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:15.754507 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a1439-fbba-4303-b643-b2da807e1ee1 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:15.807821 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-c72a1439-fbba-4303-b643-b2da807e1ee1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eeb435c645d3464c990a2427e22d922f'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:15.817110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-c72a1439-fbba-4303-b643-b2da807e1ee1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eeb435c645d3464c990a2427e22d922f'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:15.827614 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-c72a1439-fbba-4303-b643-b2da807e1ee1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eeb435c645d3464c990a2427e22d922f'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:15.828476 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-c72a1439-fbba-4303-b643-b2da807e1ee1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eeb435c645d3464c990a2427e22d922f'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:15.829226 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-c72a1439-fbba-4303-b643-b2da807e1ee1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eeb435c645d3464c990a2427e22d922f'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:15.856505 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1351/2697] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 23:00:15 2022] DELETE /identity/v3/users/eeb435c645d3464c990a2427e22d922f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:15.861501 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8b69c-a81e-44fd-acf0-0c2e682146c6 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:15.861979 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8b69c-a81e-44fd-acf0-0c2e682146c6 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:15.862113 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8b69c-a81e-44fd-acf0-0c2e682146c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:15.890081 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-6db8b69c-a81e-44fd-acf0-0c2e682146c6 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:15.892342 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-6db8b69c-a81e-44fd-acf0-0c2e682146c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:15.960095 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1347/2698] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:15.970438 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a1439-fbba-4303-b643-b2da807e1ee1 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:15.971948 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-c72a1439-fbba-4303-b643-b2da807e1ee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:16.034949 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a1439-fbba-4303-b643-b2da807e1ee1 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:16.039814 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78389e-368b-4f98-aecc-e6b96eee7fe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:16.040679 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78389e-368b-4f98-aecc-e6b96eee7fe5 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:16.040993 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78389e-368b-4f98-aecc-e6b96eee7fe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:16.043400 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78389e-368b-4f98-aecc-e6b96eee7fe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:16.057921 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78389e-368b-4f98-aecc-e6b96eee7fe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:16.059583 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78389e-368b-4f98-aecc-e6b96eee7fe5 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:16.070445 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1352/2699] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:00:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:00:16.353562 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8b69c-a81e-44fd-acf0-0c2e682146c6 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:16.355724 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-6db8b69c-a81e-44fd-acf0-0c2e682146c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:16.371516 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8b69c-a81e-44fd-acf0-0c2e682146c6 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:16.376303 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770eb17c-9310-4752-b5ba-988a2e3c8395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:16.376994 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770eb17c-9310-4752-b5ba-988a2e3c8395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:16.377210 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770eb17c-9310-4752-b5ba-988a2e3c8395 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:16.377514 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770eb17c-9310-4752-b5ba-988a2e3c8395 admin admin] PATH_INFO: `/v3/projects/8edcd6094aa84b27a37b03d4c6aa3bf3` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:16.378635 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770eb17c-9310-4752-b5ba-988a2e3c8395 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8edcd6094aa84b27a37b03d4c6aa3bf3)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:16.380550 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770eb17c-9310-4752-b5ba-988a2e3c8395 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:16.428516 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:16.430164 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-770eb17c-9310-4752-b5ba-988a2e3c8395 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8edcd6094aa84b27a37b03d4c6aa3bf3', 'request_id': 'req-770eb17c-9310-4752-b5ba-988a2e3c8395'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:16.440051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-770eb17c-9310-4752-b5ba-988a2e3c8395 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8edcd6094aa84b27a37b03d4c6aa3bf3', 'request_id': 'req-770eb17c-9310-4752-b5ba-988a2e3c8395'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:16.446544 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-770eb17c-9310-4752-b5ba-988a2e3c8395 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8edcd6094aa84b27a37b03d4c6aa3bf3', 'request_id': 'req-770eb17c-9310-4752-b5ba-988a2e3c8395'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:16.458017 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-770eb17c-9310-4752-b5ba-988a2e3c8395 admin admin] The token cache is being invalidate because project 8edcd6094aa84b27a37b03d4c6aa3bf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:00:16.458305 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-770eb17c-9310-4752-b5ba-988a2e3c8395 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:16.458967 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-770eb17c-9310-4752-b5ba-988a2e3c8395 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:16.460448 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1348/2700] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 23:00:16 2022] DELETE /identity/v3/projects/8edcd6094aa84b27a37b03d4c6aa3bf3 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:16.464603 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78389e-368b-4f98-aecc-e6b96eee7fe5 service neutron] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:16.474359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78389e-368b-4f98-aecc-e6b96eee7fe5 service neutron] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:16.520235 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:16.534701 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1cabe7-79d0-48b5-ab66-cec43ebf9eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:16.535336 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1cabe7-79d0-48b5-ab66-cec43ebf9eef admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:16.535620 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1cabe7-79d0-48b5-ab66-cec43ebf9eef admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:16.535897 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1cabe7-79d0-48b5-ab66-cec43ebf9eef admin admin] PATH_INFO: `/v3/users/72a9f5b3810d4f6fb40655084aee1e5a` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:16.537149 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1cabe7-79d0-48b5-ab66-cec43ebf9eef admin admin] RBAC: Authorizing `identity:delete_user(user_id=72a9f5b3810d4f6fb40655084aee1e5a)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:16.539125 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1cabe7-79d0-48b5-ab66-cec43ebf9eef admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:16.593754 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-ea1cabe7-79d0-48b5-ab66-cec43ebf9eef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72a9f5b3810d4f6fb40655084aee1e5a'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:16.601097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-ea1cabe7-79d0-48b5-ab66-cec43ebf9eef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72a9f5b3810d4f6fb40655084aee1e5a'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:16.608719 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-ea1cabe7-79d0-48b5-ab66-cec43ebf9eef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72a9f5b3810d4f6fb40655084aee1e5a'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:16.609404 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-ea1cabe7-79d0-48b5-ab66-cec43ebf9eef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72a9f5b3810d4f6fb40655084aee1e5a'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:16.610011 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-ea1cabe7-79d0-48b5-ab66-cec43ebf9eef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72a9f5b3810d4f6fb40655084aee1e5a'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:16.627090 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1353/2701] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 23:00:16 2022] DELETE /identity/v3/users/72a9f5b3810d4f6fb40655084aee1e5a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:16.922736 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770eb17c-9310-4752-b5ba-988a2e3c8395 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:16.924307 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-770eb17c-9310-4752-b5ba-988a2e3c8395 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:16.946317 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770eb17c-9310-4752-b5ba-988a2e3c8395 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:17.006362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fbb119-f37a-44cd-80cf-804285e3bd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:17.007036 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbb119-f37a-44cd-80cf-804285e3bd6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:17.007483 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbb119-f37a-44cd-80cf-804285e3bd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:17.007848 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbb119-f37a-44cd-80cf-804285e3bd6a admin admin] PATH_INFO: `/v3/projects/585014d761af44dbb8ff1697288f4e3f` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:17.009210 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fbb119-f37a-44cd-80cf-804285e3bd6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=585014d761af44dbb8ff1697288f4e3f)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:17.012267 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fbb119-f37a-44cd-80cf-804285e3bd6a admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:17.063485 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-05fbb119-f37a-44cd-80cf-804285e3bd6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '585014d761af44dbb8ff1697288f4e3f', 'request_id': 'req-05fbb119-f37a-44cd-80cf-804285e3bd6a'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:17.071286 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-05fbb119-f37a-44cd-80cf-804285e3bd6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '585014d761af44dbb8ff1697288f4e3f', 'request_id': 'req-05fbb119-f37a-44cd-80cf-804285e3bd6a'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:17.077640 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-05fbb119-f37a-44cd-80cf-804285e3bd6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '585014d761af44dbb8ff1697288f4e3f', 'request_id': 'req-05fbb119-f37a-44cd-80cf-804285e3bd6a'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:17.089520 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-05fbb119-f37a-44cd-80cf-804285e3bd6a admin admin] The token cache is being invalidate because project 585014d761af44dbb8ff1697288f4e3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:00:17.089927 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-05fbb119-f37a-44cd-80cf-804285e3bd6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:17.090570 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-05fbb119-f37a-44cd-80cf-804285e3bd6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:17.092060 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1349/2702] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 23:00:16 2022] DELETE /identity/v3/projects/585014d761af44dbb8ff1697288f4e3f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:17.096964 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae9e05-21c3-4c6a-aa78-cb5616cc46c8 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:17.097470 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae9e05-21c3-4c6a-aa78-cb5616cc46c8 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:17.097570 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae9e05-21c3-4c6a-aa78-cb5616cc46c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:17.119187 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-91ae9e05-21c3-4c6a-aa78-cb5616cc46c8 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:17.120626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-91ae9e05-21c3-4c6a-aa78-cb5616cc46c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:17.185268 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1354/2703] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:17.194260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af0d54-5470-4233-9914-e27be6ad1f7a None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:17.194539 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af0d54-5470-4233-9914-e27be6ad1f7a None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:17.194810 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af0d54-5470-4233-9914-e27be6ad1f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:17.214936 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-c7af0d54-5470-4233-9914-e27be6ad1f7a None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:17.216199 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-c7af0d54-5470-4233-9914-e27be6ad1f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:17.224133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1350/2704] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:17.228229 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae9e05-21c3-4c6a-aa78-cb5616cc46c8 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:17.241568 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae9e05-21c3-4c6a-aa78-cb5616cc46c8 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:17.247449 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a5a50-78ad-4127-b7b7-0027c511b641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:17.248061 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a5a50-78ad-4127-b7b7-0027c511b641 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:17.248343 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a5a50-78ad-4127-b7b7-0027c511b641 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:17.248645 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a5a50-78ad-4127-b7b7-0027c511b641 admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:17.249318 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a5a50-78ad-4127-b7b7-0027c511b641 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:00:17.249628 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a5a50-78ad-4127-b7b7-0027c511b641 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:17.251221 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a5a50-78ad-4127-b7b7-0027c511b641 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:17.266946 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1355/2705] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 23:00:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:00:17.268031 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af0d54-5470-4233-9914-e27be6ad1f7a None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:17.277798 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af0d54-5470-4233-9914-e27be6ad1f7a None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:17.284724 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e786b12-795f-471c-bd77-5752bfcc946e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:17.285353 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e786b12-795f-471c-bd77-5752bfcc946e admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:17.285635 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e786b12-795f-471c-bd77-5752bfcc946e admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:17.285905 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e786b12-795f-471c-bd77-5752bfcc946e admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:17.286543 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e786b12-795f-471c-bd77-5752bfcc946e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:17.288437 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e786b12-795f-471c-bd77-5752bfcc946e admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:17.316977 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1351/2706] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 23:00:17 2022] POST /identity/v3/projects => generated 409 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:00:17.321130 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a5a50-78ad-4127-b7b7-0027c511b641 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:17.332353 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a5a50-78ad-4127-b7b7-0027c511b641 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:17.382266 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:17.390354 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b83fe-63d4-4244-a70a-2ea878a67cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:17.391158 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b83fe-63d4-4244-a70a-2ea878a67cdc admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:17.391822 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b83fe-63d4-4244-a70a-2ea878a67cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:17.392048 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b83fe-63d4-4244-a70a-2ea878a67cdc admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:17.392773 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354b83fe-63d4-4244-a70a-2ea878a67cdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:17.394865 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354b83fe-63d4-4244-a70a-2ea878a67cdc admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:17.420256 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1356/2707] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 23:00:17 2022] POST /identity/v3/users => generated 305 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:00:17.423902 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e786b12-795f-471c-bd77-5752bfcc946e admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:17.436381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e786b12-795f-471c-bd77-5752bfcc946e admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:17.443577 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abb65d-75c2-4a9c-9569-bacfa0037be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:17.444215 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abb65d-75c2-4a9c-9569-bacfa0037be5 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:17.445125 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abb65d-75c2-4a9c-9569-bacfa0037be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:17.445443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abb65d-75c2-4a9c-9569-bacfa0037be5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:17.446079 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22abb65d-75c2-4a9c-9569-bacfa0037be5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:17.447765 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22abb65d-75c2-4a9c-9569-bacfa0037be5 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:17.457970 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1352/2708] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:00:17.462164 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b83fe-63d4-4244-a70a-2ea878a67cdc admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:17.472589 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b83fe-63d4-4244-a70a-2ea878a67cdc admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:17.479558 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42375da-d743-4531-b6ff-14409eab0639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:17.479558 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42375da-d743-4531-b6ff-14409eab0639 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:17.479558 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42375da-d743-4531-b6ff-14409eab0639 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:17.479558 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42375da-d743-4531-b6ff-14409eab0639 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:17.480097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42375da-d743-4531-b6ff-14409eab0639 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:17.481238 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42375da-d743-4531-b6ff-14409eab0639 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:17.491828 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1357/2709] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:00:17.496293 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abb65d-75c2-4a9c-9569-bacfa0037be5 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:17.506135 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abb65d-75c2-4a9c-9569-bacfa0037be5 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:17.511200 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bed657-81af-4891-8d4b-9a55b838ed24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:17.511859 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bed657-81af-4891-8d4b-9a55b838ed24 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:17.512136 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bed657-81af-4891-8d4b-9a55b838ed24 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:17.512417 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bed657-81af-4891-8d4b-9a55b838ed24 admin admin] PATH_INFO: `/v3/projects/c3fe755c19e54179b1ff819b2110a09e/users/6d2075e90a53456aaae7413ecd33fd19/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:17.528303 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bed657-81af-4891-8d4b-9a55b838ed24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3fe755c19e54179b1ff819b2110a09e, user_id=6d2075e90a53456aaae7413ecd33fd19, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:17.531000 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bed657-81af-4891-8d4b-9a55b838ed24 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:17.545610 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1353/2710] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 23:00:17 2022] PUT /identity/v3/projects/c3fe755c19e54179b1ff819b2110a09e/users/6d2075e90a53456aaae7413ecd33fd19/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:00:17.555902 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed1e9f-4ce7-4b2e-9f63-c63e926b727c None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:17.556194 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed1e9f-4ce7-4b2e-9f63-c63e926b727c None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:17.556489 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed1e9f-4ce7-4b2e-9f63-c63e926b727c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:17.583974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-8eed1e9f-4ce7-4b2e-9f63-c63e926b727c None None] MFA Rules not processed for user `6d2075e90a53456aaae7413ecd33fd19`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:17.585231 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-8eed1e9f-4ce7-4b2e-9f63-c63e926b727c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:17.655197 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1358/2711] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:17 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:17.659747 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bed657-81af-4891-8d4b-9a55b838ed24 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:17.664348 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:17.727047 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bed657-81af-4891-8d4b-9a55b838ed24 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:17.732763 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff905ca-b342-455a-8094-797d160c9c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:17.733408 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff905ca-b342-455a-8094-797d160c9c0d admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:17.733722 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff905ca-b342-455a-8094-797d160c9c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:17.733997 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff905ca-b342-455a-8094-797d160c9c0d admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:17.734620 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff905ca-b342-455a-8094-797d160c9c0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:17.736541 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff905ca-b342-455a-8094-797d160c9c0d admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:17.764575 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1354/2712] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 23:00:17 2022] POST /identity/v3/projects => generated 407 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:00:17.768698 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed1e9f-4ce7-4b2e-9f63-c63e926b727c None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:17.782827 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed1e9f-4ce7-4b2e-9f63-c63e926b727c None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:17.849057 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00811010-da64-41d4-b049-9204e600bad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:17.849717 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00811010-da64-41d4-b049-9204e600bad8 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:17.850001 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00811010-da64-41d4-b049-9204e600bad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:17.850295 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00811010-da64-41d4-b049-9204e600bad8 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:17.851121 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00811010-da64-41d4-b049-9204e600bad8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:17.852825 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00811010-da64-41d4-b049-9204e600bad8 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:17.875593 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1359/2713] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 23:00:17 2022] POST /identity/v3/users => generated 310 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:00:17.878253 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff905ca-b342-455a-8094-797d160c9c0d admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:17.890591 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff905ca-b342-455a-8094-797d160c9c0d admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:17.896025 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf01dc-6181-4abd-a3e7-590b416bf1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:17.896633 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf01dc-6181-4abd-a3e7-590b416bf1b0 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:17.897019 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf01dc-6181-4abd-a3e7-590b416bf1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:17.897179 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf01dc-6181-4abd-a3e7-590b416bf1b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:17.897867 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbf01dc-6181-4abd-a3e7-590b416bf1b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:17.899457 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbf01dc-6181-4abd-a3e7-590b416bf1b0 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:17.924206 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1355/2714] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:00:17.928626 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00811010-da64-41d4-b049-9204e600bad8 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:17.952109 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00811010-da64-41d4-b049-9204e600bad8 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:17.957138 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501287d4-75e9-4d9a-b0b3-433665343365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:17.957833 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501287d4-75e9-4d9a-b0b3-433665343365 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:17.958138 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501287d4-75e9-4d9a-b0b3-433665343365 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:17.958415 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501287d4-75e9-4d9a-b0b3-433665343365 admin admin] PATH_INFO: `/v3/projects/c0afd24e28b744d783886c56af31d5dd/users/7374cd2bf38542abbc656b16b44e557b/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:17.975062 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501287d4-75e9-4d9a-b0b3-433665343365 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0afd24e28b744d783886c56af31d5dd, user_id=7374cd2bf38542abbc656b16b44e557b, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:17.977773 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501287d4-75e9-4d9a-b0b3-433665343365 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:17.995605 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1360/2715] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 23:00:17 2022] PUT /identity/v3/projects/c0afd24e28b744d783886c56af31d5dd/users/7374cd2bf38542abbc656b16b44e557b/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:17.999806 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf01dc-6181-4abd-a3e7-590b416bf1b0 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:18.010779 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf01dc-6181-4abd-a3e7-590b416bf1b0 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:18.074174 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa388a77-d990-4a6c-b05c-7ae81be8dea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:18.075037 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa388a77-d990-4a6c-b05c-7ae81be8dea8 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:18.075321 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa388a77-d990-4a6c-b05c-7ae81be8dea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:18.075661 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa388a77-d990-4a6c-b05c-7ae81be8dea8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:18.076488 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa388a77-d990-4a6c-b05c-7ae81be8dea8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:18.078524 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa388a77-d990-4a6c-b05c-7ae81be8dea8 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:18.093878 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1356/2716] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:00:18.098284 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501287d4-75e9-4d9a-b0b3-433665343365 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:18.110567 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501287d4-75e9-4d9a-b0b3-433665343365 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:18.116546 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c6e996-db28-4a73-999c-dfe1ec3ca523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:18.117751 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c6e996-db28-4a73-999c-dfe1ec3ca523 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:18.117877 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c6e996-db28-4a73-999c-dfe1ec3ca523 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:18.118183 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c6e996-db28-4a73-999c-dfe1ec3ca523 admin admin] PATH_INFO: `/v3/projects/c0afd24e28b744d783886c56af31d5dd/users/7374cd2bf38542abbc656b16b44e557b/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:18.121630 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c6e996-db28-4a73-999c-dfe1ec3ca523 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0afd24e28b744d783886c56af31d5dd, user_id=7374cd2bf38542abbc656b16b44e557b, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:18.125327 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c6e996-db28-4a73-999c-dfe1ec3ca523 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:18.138154 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1361/2717] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 23:00:18 2022] PUT /identity/v3/projects/c0afd24e28b744d783886c56af31d5dd/users/7374cd2bf38542abbc656b16b44e557b/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:18.148585 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ec953-8977-4b35-9b63-7ac8ac49e555 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:18.148810 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ec953-8977-4b35-9b63-7ac8ac49e555 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:18.149086 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ec953-8977-4b35-9b63-7ac8ac49e555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:18.179812 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-552ec953-8977-4b35-9b63-7ac8ac49e555 None None] MFA Rules not processed for user `7374cd2bf38542abbc656b16b44e557b`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:18.181435 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-552ec953-8977-4b35-9b63-7ac8ac49e555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:18.251185 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:18.257243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1357/2718] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:18 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:18.268700 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c6e996-db28-4a73-999c-dfe1ec3ca523 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:18.270680 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-77c6e996-db28-4a73-999c-dfe1ec3ca523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:18.289386 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:18.332300 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c6e996-db28-4a73-999c-dfe1ec3ca523 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:18.336376 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab07884-3dcc-4252-8256-9cdff3a84405 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:18.337005 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab07884-3dcc-4252-8256-9cdff3a84405 service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:18.337299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab07884-3dcc-4252-8256-9cdff3a84405 service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:18.337591 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab07884-3dcc-4252-8256-9cdff3a84405 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:18.347664 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab07884-3dcc-4252-8256-9cdff3a84405 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:18.349157 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab07884-3dcc-4252-8256-9cdff3a84405 service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:18.359341 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1362/2719] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:00:18 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:00:18.509783 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ec953-8977-4b35-9b63-7ac8ac49e555 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:18.519350 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ec953-8977-4b35-9b63-7ac8ac49e555 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:18.522885 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d0e3e-2f91-489a-9f3c-04dd875d65f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:18.523502 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d0e3e-2f91-489a-9f3c-04dd875d65f9 service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:18.523773 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d0e3e-2f91-489a-9f3c-04dd875d65f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:18.524044 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d0e3e-2f91-489a-9f3c-04dd875d65f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:18.581808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5d0e3e-2f91-489a-9f3c-04dd875d65f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:18.583257 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5d0e3e-2f91-489a-9f3c-04dd875d65f9 service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:18.592298 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1358/2720] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:00:18 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:00:18.975159 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab07884-3dcc-4252-8256-9cdff3a84405 service cinder] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:18.976568 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-cab07884-3dcc-4252-8256-9cdff3a84405 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:19.041150 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab07884-3dcc-4252-8256-9cdff3a84405 service cinder] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:19.046745 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3105768-df0c-416a-a6e7-5d93054c8d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:19.047406 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3105768-df0c-416a-a6e7-5d93054c8d83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:19.047690 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3105768-df0c-416a-a6e7-5d93054c8d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:19.047969 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3105768-df0c-416a-a6e7-5d93054c8d83 admin admin] PATH_INFO: `/v3/users/b29fd534f32744f7897be562580bd96e` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:19.049096 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3105768-df0c-416a-a6e7-5d93054c8d83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b29fd534f32744f7897be562580bd96e)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:19.051111 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3105768-df0c-416a-a6e7-5d93054c8d83 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:19.099098 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-b3105768-df0c-416a-a6e7-5d93054c8d83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b29fd534f32744f7897be562580bd96e'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:19.106896 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-b3105768-df0c-416a-a6e7-5d93054c8d83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b29fd534f32744f7897be562580bd96e'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:19.113865 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-b3105768-df0c-416a-a6e7-5d93054c8d83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b29fd534f32744f7897be562580bd96e'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:19.114484 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-b3105768-df0c-416a-a6e7-5d93054c8d83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b29fd534f32744f7897be562580bd96e'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:19.115083 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-b3105768-df0c-416a-a6e7-5d93054c8d83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b29fd534f32744f7897be562580bd96e'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:19.132431 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1363/2721] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 23:00:18 2022] DELETE /identity/v3/users/b29fd534f32744f7897be562580bd96e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:19.137069 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3fb41-9594-44d4-910d-0e7429cb308d None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:19.137566 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3fb41-9594-44d4-910d-0e7429cb308d None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:19.137747 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3fb41-9594-44d4-910d-0e7429cb308d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:19.158252 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-ccd3fb41-9594-44d4-910d-0e7429cb308d None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:19.159476 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-ccd3fb41-9594-44d4-910d-0e7429cb308d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:19.223271 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1359/2722] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:19.237699 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3105768-df0c-416a-a6e7-5d93054c8d83 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:19.239346 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-b3105768-df0c-416a-a6e7-5d93054c8d83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:19.288109 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:19.305616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3105768-df0c-416a-a6e7-5d93054c8d83 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:19.309562 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1582f00d-a99b-46c9-b338-9b42c3ca4fe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:19.310167 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582f00d-a99b-46c9-b338-9b42c3ca4fe3 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:19.310446 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582f00d-a99b-46c9-b338-9b42c3ca4fe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:19.310718 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582f00d-a99b-46c9-b338-9b42c3ca4fe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:19.319580 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1582f00d-a99b-46c9-b338-9b42c3ca4fe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:19.321013 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1582f00d-a99b-46c9-b338-9b42c3ca4fe3 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:19.331693 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1364/2723] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:00:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:00:19.600617 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3fb41-9594-44d4-910d-0e7429cb308d None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:19.602293 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-ccd3fb41-9594-44d4-910d-0e7429cb308d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:19.615904 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3fb41-9594-44d4-910d-0e7429cb308d None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:19.621128 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a0347-0b96-4db6-b24e-c349148845ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:19.621914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a0347-0b96-4db6-b24e-c349148845ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:19.622193 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a0347-0b96-4db6-b24e-c349148845ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:19.622631 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a0347-0b96-4db6-b24e-c349148845ce admin admin] PATH_INFO: `/v3/projects/a220bb3708334387872ed7839dbba0d1` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:19.623832 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3a0347-0b96-4db6-b24e-c349148845ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=a220bb3708334387872ed7839dbba0d1)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:19.626059 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3a0347-0b96-4db6-b24e-c349148845ce admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:19.677821 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-fe3a0347-0b96-4db6-b24e-c349148845ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a220bb3708334387872ed7839dbba0d1', 'request_id': 'req-fe3a0347-0b96-4db6-b24e-c349148845ce'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:19.685489 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-fe3a0347-0b96-4db6-b24e-c349148845ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a220bb3708334387872ed7839dbba0d1', 'request_id': 'req-fe3a0347-0b96-4db6-b24e-c349148845ce'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:19.691475 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-fe3a0347-0b96-4db6-b24e-c349148845ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a220bb3708334387872ed7839dbba0d1', 'request_id': 'req-fe3a0347-0b96-4db6-b24e-c349148845ce'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:19.702614 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-fe3a0347-0b96-4db6-b24e-c349148845ce admin admin] The token cache is being invalidate because project a220bb3708334387872ed7839dbba0d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:00:19.702899 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-fe3a0347-0b96-4db6-b24e-c349148845ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:19.703534 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-fe3a0347-0b96-4db6-b24e-c349148845ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:19.705011 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1360/2724] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 23:00:19 2022] DELETE /identity/v3/projects/a220bb3708334387872ed7839dbba0d1 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:24.594590 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1582f00d-a99b-46c9-b338-9b42c3ca4fe3 service neutron] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:24.596056 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-1582f00d-a99b-46c9-b338-9b42c3ca4fe3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:24.609525 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1582f00d-a99b-46c9-b338-9b42c3ca4fe3 service neutron] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:24.666331 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cafe680-ee40-4e0e-9a96-a464ea696abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:24.667287 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafe680-ee40-4e0e-9a96-a464ea696abc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:24.667748 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafe680-ee40-4e0e-9a96-a464ea696abc admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:24.668019 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafe680-ee40-4e0e-9a96-a464ea696abc admin admin] PATH_INFO: `/v3/users/a65fea80f43a4f449001e09dcd632339` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:24.669174 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cafe680-ee40-4e0e-9a96-a464ea696abc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a65fea80f43a4f449001e09dcd632339)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:24.671131 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cafe680-ee40-4e0e-9a96-a464ea696abc admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:24.717159 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-0cafe680-ee40-4e0e-9a96-a464ea696abc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a65fea80f43a4f449001e09dcd632339'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:24.724501 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-0cafe680-ee40-4e0e-9a96-a464ea696abc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a65fea80f43a4f449001e09dcd632339'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:24.732290 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-0cafe680-ee40-4e0e-9a96-a464ea696abc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a65fea80f43a4f449001e09dcd632339'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:24.732930 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-0cafe680-ee40-4e0e-9a96-a464ea696abc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a65fea80f43a4f449001e09dcd632339'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:24.733572 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-0cafe680-ee40-4e0e-9a96-a464ea696abc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a65fea80f43a4f449001e09dcd632339'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:24.751369 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1365/2725] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 23:00:24 2022] DELETE /identity/v3/users/a65fea80f43a4f449001e09dcd632339 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:00:24.755827 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15999e02-8e29-4049-b2be-71590c0cf1a8 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:24.756343 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15999e02-8e29-4049-b2be-71590c0cf1a8 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:24.756473 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15999e02-8e29-4049-b2be-71590c0cf1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:24.775991 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-15999e02-8e29-4049-b2be-71590c0cf1a8 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:24.777961 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-15999e02-8e29-4049-b2be-71590c0cf1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:24.790544 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:24.838299 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1361/2726] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:24.848296 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cafe680-ee40-4e0e-9a96-a464ea696abc admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:24.849762 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-0cafe680-ee40-4e0e-9a96-a464ea696abc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:24.907158 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cafe680-ee40-4e0e-9a96-a464ea696abc admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:24.910897 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85e336d-49cc-42d3-9a61-5d96d5c11c21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:24.911499 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e336d-49cc-42d3-9a61-5d96d5c11c21 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:24.911779 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e336d-49cc-42d3-9a61-5d96d5c11c21 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:24.912050 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e336d-49cc-42d3-9a61-5d96d5c11c21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:24.920962 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85e336d-49cc-42d3-9a61-5d96d5c11c21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:24.922427 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85e336d-49cc-42d3-9a61-5d96d5c11c21 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:24.931867 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1366/2727] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:00:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:00:25.217501 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15999e02-8e29-4049-b2be-71590c0cf1a8 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:25.219938 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-15999e02-8e29-4049-b2be-71590c0cf1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:25.233487 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15999e02-8e29-4049-b2be-71590c0cf1a8 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:25.238818 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214298c-864c-4f02-926b-0026d12770ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:25.239465 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214298c-864c-4f02-926b-0026d12770ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:25.239729 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214298c-864c-4f02-926b-0026d12770ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:25.240005 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214298c-864c-4f02-926b-0026d12770ae admin admin] PATH_INFO: `/v3/projects/4fa99c4c227e4b1289e9f5da1db3ff02` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:25.241342 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3214298c-864c-4f02-926b-0026d12770ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fa99c4c227e4b1289e9f5da1db3ff02)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:25.243498 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3214298c-864c-4f02-926b-0026d12770ae admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:25.301443 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-3214298c-864c-4f02-926b-0026d12770ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fa99c4c227e4b1289e9f5da1db3ff02', 'request_id': 'req-3214298c-864c-4f02-926b-0026d12770ae'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:25.309665 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-3214298c-864c-4f02-926b-0026d12770ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fa99c4c227e4b1289e9f5da1db3ff02', 'request_id': 'req-3214298c-864c-4f02-926b-0026d12770ae'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:25.315797 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-3214298c-864c-4f02-926b-0026d12770ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fa99c4c227e4b1289e9f5da1db3ff02', 'request_id': 'req-3214298c-864c-4f02-926b-0026d12770ae'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:25.327968 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-3214298c-864c-4f02-926b-0026d12770ae admin admin] The token cache is being invalidate because project 4fa99c4c227e4b1289e9f5da1db3ff02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:00:25.328256 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-3214298c-864c-4f02-926b-0026d12770ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:25.329100 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-3214298c-864c-4f02-926b-0026d12770ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:25.330685 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1362/2728] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 23:00:25 2022] DELETE /identity/v3/projects/4fa99c4c227e4b1289e9f5da1db3ff02 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:25.335826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bdfbe3-c8a4-4883-82f3-9e4fec70ee05 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:25.336320 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bdfbe3-c8a4-4883-82f3-9e4fec70ee05 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:25.336624 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bdfbe3-c8a4-4883-82f3-9e4fec70ee05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:25.345038 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:25.359131 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-f6bdfbe3-c8a4-4883-82f3-9e4fec70ee05 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:25.362035 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-f6bdfbe3-c8a4-4883-82f3-9e4fec70ee05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:25.426161 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1367/2729] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:25.434551 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a036c-cb67-475f-be57-8ca3fb0ef9bf None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:25.434863 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a036c-cb67-475f-be57-8ca3fb0ef9bf None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:25.435139 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a036c-cb67-475f-be57-8ca3fb0ef9bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:25.455394 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-5c7a036c-cb67-475f-be57-8ca3fb0ef9bf None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:25.456603 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-5c7a036c-cb67-475f-be57-8ca3fb0ef9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:25.464678 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1363/2730] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:25.469433 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bdfbe3-c8a4-4883-82f3-9e4fec70ee05 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:25.479185 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bdfbe3-c8a4-4883-82f3-9e4fec70ee05 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:25.485126 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb7757-5491-4002-bca2-cdacf2a00b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:25.485775 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb7757-5491-4002-bca2-cdacf2a00b49 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:25.486055 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb7757-5491-4002-bca2-cdacf2a00b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:25.486349 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb7757-5491-4002-bca2-cdacf2a00b49 admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:25.486974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bb7757-5491-4002-bca2-cdacf2a00b49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:00:25.487275 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bb7757-5491-4002-bca2-cdacf2a00b49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:25.488873 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bb7757-5491-4002-bca2-cdacf2a00b49 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:25.502055 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1368/2731] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 23:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:00:25.506448 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7a036c-cb67-475f-be57-8ca3fb0ef9bf None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:25.516969 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7a036c-cb67-475f-be57-8ca3fb0ef9bf None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:25.521749 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0a831-ce58-4922-92b0-8c88e0158e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:25.522350 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0a831-ce58-4922-92b0-8c88e0158e97 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:25.522631 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0a831-ce58-4922-92b0-8c88e0158e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:25.522906 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0a831-ce58-4922-92b0-8c88e0158e97 admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:25.523536 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d0a831-ce58-4922-92b0-8c88e0158e97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:25.525612 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d0a831-ce58-4922-92b0-8c88e0158e97 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:25.555516 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1364/2732] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 23:00:25 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:00:25.560481 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb7757-5491-4002-bca2-cdacf2a00b49 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:25.575307 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb7757-5491-4002-bca2-cdacf2a00b49 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:25.638762 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049912b8-472f-40cd-9cf7-d765b02ee27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:25.639489 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049912b8-472f-40cd-9cf7-d765b02ee27a admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:25.639654 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049912b8-472f-40cd-9cf7-d765b02ee27a admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:25.639931 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049912b8-472f-40cd-9cf7-d765b02ee27a admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:25.640553 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049912b8-472f-40cd-9cf7-d765b02ee27a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:25.642758 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049912b8-472f-40cd-9cf7-d765b02ee27a admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:25.670350 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1369/2733] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 23:00:25 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:00:25.671551 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0a831-ce58-4922-92b0-8c88e0158e97 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:25.681793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0a831-ce58-4922-92b0-8c88e0158e97 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:25.686808 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185095a6-5141-4c06-96b6-0826c4d19e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:25.687398 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185095a6-5141-4c06-96b6-0826c4d19e49 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:25.687672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185095a6-5141-4c06-96b6-0826c4d19e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:25.688085 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185095a6-5141-4c06-96b6-0826c4d19e49 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:25.688571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185095a6-5141-4c06-96b6-0826c4d19e49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:25.690183 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185095a6-5141-4c06-96b6-0826c4d19e49 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:25.700737 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1365/2734] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:00:25.705051 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049912b8-472f-40cd-9cf7-d765b02ee27a admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:25.716304 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049912b8-472f-40cd-9cf7-d765b02ee27a admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:25.720962 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa840a-eca5-46b8-88b3-850a69a2febb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:25.721597 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa840a-eca5-46b8-88b3-850a69a2febb admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:25.721921 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa840a-eca5-46b8-88b3-850a69a2febb admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:25.722173 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa840a-eca5-46b8-88b3-850a69a2febb admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:25.722808 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa840a-eca5-46b8-88b3-850a69a2febb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:25.724396 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa840a-eca5-46b8-88b3-850a69a2febb admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:25.739159 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1370/2735] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:00:25.743640 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185095a6-5141-4c06-96b6-0826c4d19e49 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:25.753648 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185095a6-5141-4c06-96b6-0826c4d19e49 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:25.758207 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865bbf95-21aa-46fc-b6a7-44c989017cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:25.758848 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865bbf95-21aa-46fc-b6a7-44c989017cae admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:25.759123 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865bbf95-21aa-46fc-b6a7-44c989017cae admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:25.759390 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865bbf95-21aa-46fc-b6a7-44c989017cae admin admin] PATH_INFO: `/v3/projects/032c383b27474bb68698557ffa79f965/users/a6e64cdb87404c51b1b65d66207ba7f5/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:25.773792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865bbf95-21aa-46fc-b6a7-44c989017cae admin admin] RBAC: Authorizing `identity:create_grant(project_id=032c383b27474bb68698557ffa79f965, user_id=a6e64cdb87404c51b1b65d66207ba7f5, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:25.776963 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865bbf95-21aa-46fc-b6a7-44c989017cae admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:25.789364 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1366/2736] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 23:00:25 2022] PUT /identity/v3/projects/032c383b27474bb68698557ffa79f965/users/a6e64cdb87404c51b1b65d66207ba7f5/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:25.800514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8dd9a-1786-4acb-94c2-77e79a4e1c5e None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:25.800821 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8dd9a-1786-4acb-94c2-77e79a4e1c5e None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:25.801110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8dd9a-1786-4acb-94c2-77e79a4e1c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:25.828540 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-c9a8dd9a-1786-4acb-94c2-77e79a4e1c5e None None] MFA Rules not processed for user `a6e64cdb87404c51b1b65d66207ba7f5`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:25.829749 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-c9a8dd9a-1786-4acb-94c2-77e79a4e1c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:25.890223 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:25.897845 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1371/2737] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:25.904072 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865bbf95-21aa-46fc-b6a7-44c989017cae admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:25.957417 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:25.982708 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865bbf95-21aa-46fc-b6a7-44c989017cae admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:25.987480 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d9d45-8f7b-40bd-ad93-76c5b3f6dc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:25.988115 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d9d45-8f7b-40bd-ad93-76c5b3f6dc62 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:25.988399 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d9d45-8f7b-40bd-ad93-76c5b3f6dc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:25.988673 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d9d45-8f7b-40bd-ad93-76c5b3f6dc62 admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:25.989335 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d9d45-8f7b-40bd-ad93-76c5b3f6dc62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:25.991244 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d9d45-8f7b-40bd-ad93-76c5b3f6dc62 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:26.018019 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1367/2738] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 23:00:25 2022] POST /identity/v3/projects => generated 379 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:00:26.022164 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a8dd9a-1786-4acb-94c2-77e79a4e1c5e None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:26.033941 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a8dd9a-1786-4acb-94c2-77e79a4e1c5e None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:26.091006 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877b2a5-e8d1-492c-b748-c4c3312c4ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:26.091630 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877b2a5-e8d1-492c-b748-c4c3312c4ad5 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:26.091910 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877b2a5-e8d1-492c-b748-c4c3312c4ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:26.092188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877b2a5-e8d1-492c-b748-c4c3312c4ad5 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:26.092807 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5877b2a5-e8d1-492c-b748-c4c3312c4ad5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:26.094832 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5877b2a5-e8d1-492c-b748-c4c3312c4ad5 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:26.116737 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1372/2739] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 23:00:26 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:00:26.121693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d9d45-8f7b-40bd-ad93-76c5b3f6dc62 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:26.132116 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d9d45-8f7b-40bd-ad93-76c5b3f6dc62 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:26.137191 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685727fa-c9d2-474a-ae1c-6335179ac23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:26.137815 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685727fa-c9d2-474a-ae1c-6335179ac23d admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:26.138093 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685727fa-c9d2-474a-ae1c-6335179ac23d admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:26.138375 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685727fa-c9d2-474a-ae1c-6335179ac23d admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:26.139002 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685727fa-c9d2-474a-ae1c-6335179ac23d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:26.140596 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685727fa-c9d2-474a-ae1c-6335179ac23d admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:26.151086 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1368/2740] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:00:26.155205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877b2a5-e8d1-492c-b748-c4c3312c4ad5 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:26.165508 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877b2a5-e8d1-492c-b748-c4c3312c4ad5 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:26.171070 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f9cf4b-8667-4417-90b6-61595704db31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:26.171753 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f9cf4b-8667-4417-90b6-61595704db31 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:26.172055 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f9cf4b-8667-4417-90b6-61595704db31 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:26.172375 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f9cf4b-8667-4417-90b6-61595704db31 admin admin] PATH_INFO: `/v3/projects/456da5b5bb244dbb931d78f4b28f2d6a/users/1fb6ab963afa4c1ca14dfc17278cf67c/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:26.189812 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f9cf4b-8667-4417-90b6-61595704db31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=456da5b5bb244dbb931d78f4b28f2d6a, user_id=1fb6ab963afa4c1ca14dfc17278cf67c, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:26.192559 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f9cf4b-8667-4417-90b6-61595704db31 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:26.205725 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1373/2741] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 23:00:26 2022] PUT /identity/v3/projects/456da5b5bb244dbb931d78f4b28f2d6a/users/1fb6ab963afa4c1ca14dfc17278cf67c/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:26.211848 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685727fa-c9d2-474a-ae1c-6335179ac23d admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:26.226812 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685727fa-c9d2-474a-ae1c-6335179ac23d admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:26.291666 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168727ca-3ceb-420a-a289-d1c7bd0cc9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:26.292988 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168727ca-3ceb-420a-a289-d1c7bd0cc9a4 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:26.293387 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168727ca-3ceb-420a-a289-d1c7bd0cc9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:26.293792 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168727ca-3ceb-420a-a289-d1c7bd0cc9a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:26.294752 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168727ca-3ceb-420a-a289-d1c7bd0cc9a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:26.296514 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168727ca-3ceb-420a-a289-d1c7bd0cc9a4 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:26.308574 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1369/2742] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:00:26.312952 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f9cf4b-8667-4417-90b6-61595704db31 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:26.323814 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f9cf4b-8667-4417-90b6-61595704db31 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:26.328623 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e0da5-dde6-4b2d-9ad2-456479305e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:26.329297 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e0da5-dde6-4b2d-9ad2-456479305e04 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:26.329606 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e0da5-dde6-4b2d-9ad2-456479305e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:26.329885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e0da5-dde6-4b2d-9ad2-456479305e04 admin admin] PATH_INFO: `/v3/projects/456da5b5bb244dbb931d78f4b28f2d6a/users/1fb6ab963afa4c1ca14dfc17278cf67c/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:26.332023 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e0da5-dde6-4b2d-9ad2-456479305e04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=456da5b5bb244dbb931d78f4b28f2d6a, user_id=1fb6ab963afa4c1ca14dfc17278cf67c, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:26.334741 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e0da5-dde6-4b2d-9ad2-456479305e04 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:26.348462 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1374/2743] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 23:00:26 2022] PUT /identity/v3/projects/456da5b5bb244dbb931d78f4b28f2d6a/users/1fb6ab963afa4c1ca14dfc17278cf67c/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:26.361080 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f4e60-0f22-433d-a5a3-68a37a1aea47 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:26.361405 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f4e60-0f22-433d-a5a3-68a37a1aea47 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:26.361681 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f4e60-0f22-433d-a5a3-68a37a1aea47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:26.392404 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-5c2f4e60-0f22-433d-a5a3-68a37a1aea47 None None] MFA Rules not processed for user `1fb6ab963afa4c1ca14dfc17278cf67c`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:26.393866 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-5c2f4e60-0f22-433d-a5a3-68a37a1aea47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:26.470463 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1370/2744] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:26 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:26.482110 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e0da5-dde6-4b2d-9ad2-456479305e04 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:26.483855 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-d35e0da5-dde6-4b2d-9ad2-456479305e04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:26.544088 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e0da5-dde6-4b2d-9ad2-456479305e04 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:26.548054 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad3cdd-9d5a-48ad-b7ce-571d03d7814d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:26.548673 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad3cdd-9d5a-48ad-b7ce-571d03d7814d service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:26.548986 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad3cdd-9d5a-48ad-b7ce-571d03d7814d service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:26.549262 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad3cdd-9d5a-48ad-b7ce-571d03d7814d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:26.559647 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ad3cdd-9d5a-48ad-b7ce-571d03d7814d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:26.561084 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ad3cdd-9d5a-48ad-b7ce-571d03d7814d service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:26.572553 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1375/2745] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:00:26 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:00:26.653184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f4e60-0f22-433d-a5a3-68a37a1aea47 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:26.663044 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f4e60-0f22-433d-a5a3-68a37a1aea47 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:26.666771 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3fa6da-d838-4ca8-a546-45afdc953728 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:26.667823 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3fa6da-d838-4ca8-a546-45afdc953728 service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:26.668116 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3fa6da-d838-4ca8-a546-45afdc953728 service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:26.668465 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3fa6da-d838-4ca8-a546-45afdc953728 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:26.704498 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:26.733077 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3fa6da-d838-4ca8-a546-45afdc953728 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:26.734569 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3fa6da-d838-4ca8-a546-45afdc953728 service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:26.744218 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1371/2746] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:00:26 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:00:28.051191 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad3cdd-9d5a-48ad-b7ce-571d03d7814d service cinder] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:28.052860 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-d4ad3cdd-9d5a-48ad-b7ce-571d03d7814d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:28.112654 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:28.128502 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad3cdd-9d5a-48ad-b7ce-571d03d7814d service cinder] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:28.133655 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89779c8f-588a-4c85-8c74-9b3342738f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:28.134298 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89779c8f-588a-4c85-8c74-9b3342738f87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:28.134588 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89779c8f-588a-4c85-8c74-9b3342738f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:28.134944 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89779c8f-588a-4c85-8c74-9b3342738f87 admin admin] PATH_INFO: `/v3/users/6d2075e90a53456aaae7413ecd33fd19` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:28.136925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89779c8f-588a-4c85-8c74-9b3342738f87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d2075e90a53456aaae7413ecd33fd19)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:28.139295 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89779c8f-588a-4c85-8c74-9b3342738f87 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:28.190083 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-89779c8f-588a-4c85-8c74-9b3342738f87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d2075e90a53456aaae7413ecd33fd19'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:28.198658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-89779c8f-588a-4c85-8c74-9b3342738f87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d2075e90a53456aaae7413ecd33fd19'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:28.206157 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-89779c8f-588a-4c85-8c74-9b3342738f87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d2075e90a53456aaae7413ecd33fd19'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:28.206765 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-89779c8f-588a-4c85-8c74-9b3342738f87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d2075e90a53456aaae7413ecd33fd19'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:28.207388 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-89779c8f-588a-4c85-8c74-9b3342738f87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d2075e90a53456aaae7413ecd33fd19'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:28.226900 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1376/2747] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 23:00:28 2022] DELETE /identity/v3/users/6d2075e90a53456aaae7413ecd33fd19 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:28.232384 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba25afe-b43b-484a-960c-8d7ba5697fb5 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:28.232892 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba25afe-b43b-484a-960c-8d7ba5697fb5 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:28.233002 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba25afe-b43b-484a-960c-8d7ba5697fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:28.255234 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-fba25afe-b43b-484a-960c-8d7ba5697fb5 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:28.256817 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-fba25afe-b43b-484a-960c-8d7ba5697fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:28.330693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1372/2748] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:28.341180 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89779c8f-588a-4c85-8c74-9b3342738f87 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:28.342632 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-89779c8f-588a-4c85-8c74-9b3342738f87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:28.399670 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89779c8f-588a-4c85-8c74-9b3342738f87 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:28.404184 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951691c6-6aaf-4831-a865-d99ce9893698 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:28.404818 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951691c6-6aaf-4831-a865-d99ce9893698 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:28.405222 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951691c6-6aaf-4831-a865-d99ce9893698 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:28.405525 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951691c6-6aaf-4831-a865-d99ce9893698 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:28.414694 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951691c6-6aaf-4831-a865-d99ce9893698 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:28.416136 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951691c6-6aaf-4831-a865-d99ce9893698 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:28.429428 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1377/2749] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:00:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:00:28.705233 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba25afe-b43b-484a-960c-8d7ba5697fb5 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:28.706748 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-fba25afe-b43b-484a-960c-8d7ba5697fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:28.720864 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba25afe-b43b-484a-960c-8d7ba5697fb5 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:28.726130 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ba72e-791f-4cac-b40d-233a93290012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:28.727188 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ba72e-791f-4cac-b40d-233a93290012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:28.727819 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ba72e-791f-4cac-b40d-233a93290012 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:28.728278 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ba72e-791f-4cac-b40d-233a93290012 admin admin] PATH_INFO: `/v3/projects/c3fe755c19e54179b1ff819b2110a09e` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:28.730523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3ba72e-791f-4cac-b40d-233a93290012 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3fe755c19e54179b1ff819b2110a09e)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:28.732933 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3ba72e-791f-4cac-b40d-233a93290012 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:28.783365 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-fe3ba72e-791f-4cac-b40d-233a93290012 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3fe755c19e54179b1ff819b2110a09e', 'request_id': 'req-fe3ba72e-791f-4cac-b40d-233a93290012'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:28.790968 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-fe3ba72e-791f-4cac-b40d-233a93290012 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3fe755c19e54179b1ff819b2110a09e', 'request_id': 'req-fe3ba72e-791f-4cac-b40d-233a93290012'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:28.796623 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-fe3ba72e-791f-4cac-b40d-233a93290012 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3fe755c19e54179b1ff819b2110a09e', 'request_id': 'req-fe3ba72e-791f-4cac-b40d-233a93290012'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:28.807280 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-fe3ba72e-791f-4cac-b40d-233a93290012 admin admin] The token cache is being invalidate because project c3fe755c19e54179b1ff819b2110a09e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:00:28.808094 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-fe3ba72e-791f-4cac-b40d-233a93290012 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:28.808694 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-fe3ba72e-791f-4cac-b40d-233a93290012 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:28.810274 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1373/2750] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 23:00:28 2022] DELETE /identity/v3/projects/c3fe755c19e54179b1ff819b2110a09e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:28.814388 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951691c6-6aaf-4831-a865-d99ce9893698 service neutron] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:28.824873 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951691c6-6aaf-4831-a865-d99ce9893698 service neutron] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:28.880191 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b2e0c-e65e-4c41-885e-a8933fa6eee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:28.880840 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b2e0c-e65e-4c41-885e-a8933fa6eee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:28.881121 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b2e0c-e65e-4c41-885e-a8933fa6eee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:28.881440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b2e0c-e65e-4c41-885e-a8933fa6eee3 admin admin] PATH_INFO: `/v3/users/7374cd2bf38542abbc656b16b44e557b` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:28.882600 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4b2e0c-e65e-4c41-885e-a8933fa6eee3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7374cd2bf38542abbc656b16b44e557b)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:28.884525 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4b2e0c-e65e-4c41-885e-a8933fa6eee3 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:28.906094 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:28.938986 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-fe4b2e0c-e65e-4c41-885e-a8933fa6eee3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7374cd2bf38542abbc656b16b44e557b'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:28.946224 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-fe4b2e0c-e65e-4c41-885e-a8933fa6eee3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7374cd2bf38542abbc656b16b44e557b'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:28.954793 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-fe4b2e0c-e65e-4c41-885e-a8933fa6eee3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7374cd2bf38542abbc656b16b44e557b'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:28.955457 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-fe4b2e0c-e65e-4c41-885e-a8933fa6eee3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7374cd2bf38542abbc656b16b44e557b'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:28.956137 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-fe4b2e0c-e65e-4c41-885e-a8933fa6eee3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7374cd2bf38542abbc656b16b44e557b'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:28.974167 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1378/2751] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 23:00:28 2022] DELETE /identity/v3/users/7374cd2bf38542abbc656b16b44e557b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:29.251820 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ba72e-791f-4cac-b40d-233a93290012 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:29.253306 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-fe3ba72e-791f-4cac-b40d-233a93290012 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:29.267353 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ba72e-791f-4cac-b40d-233a93290012 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:29.320899 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:29.329104 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd44019-ea2e-455d-9fa3-ea7f70baf146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:29.330272 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd44019-ea2e-455d-9fa3-ea7f70baf146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:29.330707 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd44019-ea2e-455d-9fa3-ea7f70baf146 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:29.331096 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd44019-ea2e-455d-9fa3-ea7f70baf146 admin admin] PATH_INFO: `/v3/projects/c0afd24e28b744d783886c56af31d5dd` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:29.332324 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd44019-ea2e-455d-9fa3-ea7f70baf146 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0afd24e28b744d783886c56af31d5dd)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:29.334403 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd44019-ea2e-455d-9fa3-ea7f70baf146 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:29.384415 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-edd44019-ea2e-455d-9fa3-ea7f70baf146 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0afd24e28b744d783886c56af31d5dd', 'request_id': 'req-edd44019-ea2e-455d-9fa3-ea7f70baf146'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:29.392416 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-edd44019-ea2e-455d-9fa3-ea7f70baf146 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0afd24e28b744d783886c56af31d5dd', 'request_id': 'req-edd44019-ea2e-455d-9fa3-ea7f70baf146'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:29.398259 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-edd44019-ea2e-455d-9fa3-ea7f70baf146 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0afd24e28b744d783886c56af31d5dd', 'request_id': 'req-edd44019-ea2e-455d-9fa3-ea7f70baf146'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:29.410680 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-edd44019-ea2e-455d-9fa3-ea7f70baf146 admin admin] The token cache is being invalidate because project c0afd24e28b744d783886c56af31d5dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:00:29.410970 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-edd44019-ea2e-455d-9fa3-ea7f70baf146 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:29.411627 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-edd44019-ea2e-455d-9fa3-ea7f70baf146 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:29.413110 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1374/2752] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 23:00:29 2022] DELETE /identity/v3/projects/c0afd24e28b744d783886c56af31d5dd => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:29.419189 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31760e2-3a53-47f3-8a8c-efeabe23d4c9 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:29.419647 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31760e2-3a53-47f3-8a8c-efeabe23d4c9 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:29.419733 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31760e2-3a53-47f3-8a8c-efeabe23d4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:29.441432 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-a31760e2-3a53-47f3-8a8c-efeabe23d4c9 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:29.442751 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-a31760e2-3a53-47f3-8a8c-efeabe23d4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:29.505985 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1379/2753] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:29.515224 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29052471-3382-4167-8a3b-d9d49dc48aac None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:29.515853 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29052471-3382-4167-8a3b-d9d49dc48aac None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:29.516137 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29052471-3382-4167-8a3b-d9d49dc48aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:29.537144 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-29052471-3382-4167-8a3b-d9d49dc48aac None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:29.538482 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-29052471-3382-4167-8a3b-d9d49dc48aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:29.548646 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1375/2754] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:29.553451 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31760e2-3a53-47f3-8a8c-efeabe23d4c9 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:29.564584 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31760e2-3a53-47f3-8a8c-efeabe23d4c9 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:29.573269 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5708d79-cf5c-4520-b05b-e3969c187789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:29.573269 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5708d79-cf5c-4520-b05b-e3969c187789 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:29.573269 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5708d79-cf5c-4520-b05b-e3969c187789 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:29.573269 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5708d79-cf5c-4520-b05b-e3969c187789 admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:29.573742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5708d79-cf5c-4520-b05b-e3969c187789 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:00:29.573742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5708d79-cf5c-4520-b05b-e3969c187789 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:29.573742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5708d79-cf5c-4520-b05b-e3969c187789 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:29.591322 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1380/2755] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 23:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:00:29.592556 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29052471-3382-4167-8a3b-d9d49dc48aac None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:29.602168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29052471-3382-4167-8a3b-d9d49dc48aac None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:29.608030 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31582190-e980-4135-88ef-d9977e284437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:29.608710 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31582190-e980-4135-88ef-d9977e284437 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:29.608989 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31582190-e980-4135-88ef-d9977e284437 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:29.609262 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31582190-e980-4135-88ef-d9977e284437 admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:29.609914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31582190-e980-4135-88ef-d9977e284437 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:29.612050 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31582190-e980-4135-88ef-d9977e284437 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:29.640399 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1376/2756] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 23:00:29 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:00:29.644632 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5708d79-cf5c-4520-b05b-e3969c187789 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:29.655413 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5708d79-cf5c-4520-b05b-e3969c187789 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:29.710496 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4244ca-4834-4f54-8bf7-ae47c47ede61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:29.711104 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4244ca-4834-4f54-8bf7-ae47c47ede61 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:29.711401 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4244ca-4834-4f54-8bf7-ae47c47ede61 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:29.711658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4244ca-4834-4f54-8bf7-ae47c47ede61 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:29.712278 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4244ca-4834-4f54-8bf7-ae47c47ede61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:29.714865 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4244ca-4834-4f54-8bf7-ae47c47ede61 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:29.735883 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1381/2757] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 23:00:29 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:00:29.740605 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31582190-e980-4135-88ef-d9977e284437 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:29.751810 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31582190-e980-4135-88ef-d9977e284437 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:29.756439 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6638caaa-b8ad-40f1-acc3-f53d8cb667e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:29.757061 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638caaa-b8ad-40f1-acc3-f53d8cb667e9 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:29.757381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638caaa-b8ad-40f1-acc3-f53d8cb667e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:29.757655 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638caaa-b8ad-40f1-acc3-f53d8cb667e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:29.758291 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6638caaa-b8ad-40f1-acc3-f53d8cb667e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:29.759883 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6638caaa-b8ad-40f1-acc3-f53d8cb667e9 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:29.770404 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1377/2758] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:00:29.774295 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4244ca-4834-4f54-8bf7-ae47c47ede61 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:29.785145 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4244ca-4834-4f54-8bf7-ae47c47ede61 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:29.790036 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c571ecb-ef9e-4610-8b1d-b0c954eb68f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:29.790666 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c571ecb-ef9e-4610-8b1d-b0c954eb68f7 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:29.790949 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c571ecb-ef9e-4610-8b1d-b0c954eb68f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:29.791225 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c571ecb-ef9e-4610-8b1d-b0c954eb68f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:29.791987 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c571ecb-ef9e-4610-8b1d-b0c954eb68f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:29.793516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c571ecb-ef9e-4610-8b1d-b0c954eb68f7 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:29.798842 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:29.805121 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1382/2759] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:00:29.808968 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6638caaa-b8ad-40f1-acc3-f53d8cb667e9 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:29.818847 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6638caaa-b8ad-40f1-acc3-f53d8cb667e9 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:29.823967 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a35d7c-7178-49db-9962-eeabc67dedf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:29.824612 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a35d7c-7178-49db-9962-eeabc67dedf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:29.824887 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a35d7c-7178-49db-9962-eeabc67dedf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:29.825156 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a35d7c-7178-49db-9962-eeabc67dedf4 admin admin] PATH_INFO: `/v3/projects/2cc07f31b9354c6681368e4216732303/users/c9867310516b4fb5800665b97daac11b/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:29.840312 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a35d7c-7178-49db-9962-eeabc67dedf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cc07f31b9354c6681368e4216732303, user_id=c9867310516b4fb5800665b97daac11b, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:29.843141 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a35d7c-7178-49db-9962-eeabc67dedf4 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:29.855944 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1378/2760] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 23:00:29 2022] PUT /identity/v3/projects/2cc07f31b9354c6681368e4216732303/users/c9867310516b4fb5800665b97daac11b/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:29.866187 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba8da5-0523-40b4-b23f-79b20fb69bad None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:29.866480 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba8da5-0523-40b4-b23f-79b20fb69bad None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:29.866762 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba8da5-0523-40b4-b23f-79b20fb69bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:29.894938 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-19ba8da5-0523-40b4-b23f-79b20fb69bad None None] MFA Rules not processed for user `c9867310516b4fb5800665b97daac11b`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:29.896167 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-19ba8da5-0523-40b4-b23f-79b20fb69bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:29.961572 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1383/2761] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:29.966965 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a35d7c-7178-49db-9962-eeabc67dedf4 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:30.029305 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a35d7c-7178-49db-9962-eeabc67dedf4 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:30.033949 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c80db-d7df-460b-951f-5a07f931fa6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:30.034553 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c80db-d7df-460b-951f-5a07f931fa6c admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:30.034834 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c80db-d7df-460b-951f-5a07f931fa6c admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:30.035110 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c80db-d7df-460b-951f-5a07f931fa6c admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:30.035758 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69c80db-d7df-460b-951f-5a07f931fa6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:30.037738 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69c80db-d7df-460b-951f-5a07f931fa6c admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:30.066243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1379/2762] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 23:00:29 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:00:30.070698 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ba8da5-0523-40b4-b23f-79b20fb69bad None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:30.083263 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ba8da5-0523-40b4-b23f-79b20fb69bad None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:30.139164 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7f23c-8293-48d8-bc0d-14d96d71394f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:30.139777 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7f23c-8293-48d8-bc0d-14d96d71394f admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:30.140056 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7f23c-8293-48d8-bc0d-14d96d71394f admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:30.140328 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7f23c-8293-48d8-bc0d-14d96d71394f admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:30.140950 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7f23c-8293-48d8-bc0d-14d96d71394f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:30.142905 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7f23c-8293-48d8-bc0d-14d96d71394f admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:30.163290 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1384/2763] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 23:00:30 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:00:30.167557 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c80db-d7df-460b-951f-5a07f931fa6c admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:30.178851 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c80db-d7df-460b-951f-5a07f931fa6c admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:30.184565 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb9892d-fedb-43f4-839c-46f9d217efdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:30.185165 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9892d-fedb-43f4-839c-46f9d217efdf admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:30.185470 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9892d-fedb-43f4-839c-46f9d217efdf admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:30.185736 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9892d-fedb-43f4-839c-46f9d217efdf admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:30.186364 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb9892d-fedb-43f4-839c-46f9d217efdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:30.187938 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb9892d-fedb-43f4-839c-46f9d217efdf admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:30.190070 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:30.201501 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1380/2764] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:00:30.205242 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7f23c-8293-48d8-bc0d-14d96d71394f admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:30.216535 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7f23c-8293-48d8-bc0d-14d96d71394f admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:30.222232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc8c4d-4ded-4e40-ac52-7abc6b8bfdd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:30.223105 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc8c4d-4ded-4e40-ac52-7abc6b8bfdd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:30.223543 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc8c4d-4ded-4e40-ac52-7abc6b8bfdd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:30.223984 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc8c4d-4ded-4e40-ac52-7abc6b8bfdd5 admin admin] PATH_INFO: `/v3/projects/eb8b7858524e42b08d6f1636bbb7f64c/users/6b98f8d506bc4e6faedb79f2bd87a1ae/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:30.240057 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbc8c4d-4ded-4e40-ac52-7abc6b8bfdd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb8b7858524e42b08d6f1636bbb7f64c, user_id=6b98f8d506bc4e6faedb79f2bd87a1ae, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:30.243018 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbc8c4d-4ded-4e40-ac52-7abc6b8bfdd5 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:30.256412 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1385/2765] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 23:00:30 2022] PUT /identity/v3/projects/eb8b7858524e42b08d6f1636bbb7f64c/users/6b98f8d506bc4e6faedb79f2bd87a1ae/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:30.263692 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb9892d-fedb-43f4-839c-46f9d217efdf admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:30.272103 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb9892d-fedb-43f4-839c-46f9d217efdf admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:30.330631 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c38f9c8-2adc-4746-b7fb-a2e8a89882b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:30.331127 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38f9c8-2adc-4746-b7fb-a2e8a89882b4 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:30.331404 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38f9c8-2adc-4746-b7fb-a2e8a89882b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:30.331673 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38f9c8-2adc-4746-b7fb-a2e8a89882b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:30.332295 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c38f9c8-2adc-4746-b7fb-a2e8a89882b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:30.333912 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c38f9c8-2adc-4746-b7fb-a2e8a89882b4 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:30.344449 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1381/2766] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:00:30.348699 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc8c4d-4ded-4e40-ac52-7abc6b8bfdd5 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:30.360071 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc8c4d-4ded-4e40-ac52-7abc6b8bfdd5 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:30.364873 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275ab4f-72a8-4554-ba7d-978b57cb7c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:30.365566 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275ab4f-72a8-4554-ba7d-978b57cb7c7c admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:30.365849 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275ab4f-72a8-4554-ba7d-978b57cb7c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:30.366125 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275ab4f-72a8-4554-ba7d-978b57cb7c7c admin admin] PATH_INFO: `/v3/projects/eb8b7858524e42b08d6f1636bbb7f64c/users/6b98f8d506bc4e6faedb79f2bd87a1ae/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:30.368342 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8275ab4f-72a8-4554-ba7d-978b57cb7c7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb8b7858524e42b08d6f1636bbb7f64c, user_id=6b98f8d506bc4e6faedb79f2bd87a1ae, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:30.371520 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8275ab4f-72a8-4554-ba7d-978b57cb7c7c admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:30.381742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1386/2767] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 23:00:30 2022] PUT /identity/v3/projects/eb8b7858524e42b08d6f1636bbb7f64c/users/6b98f8d506bc4e6faedb79f2bd87a1ae/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:30.450432 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2f5e8-ff16-4bb7-a7d5-0f28d922eacb None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:30.450721 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2f5e8-ff16-4bb7-a7d5-0f28d922eacb None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:30.450998 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2f5e8-ff16-4bb7-a7d5-0f28d922eacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:30.478875 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-66e2f5e8-ff16-4bb7-a7d5-0f28d922eacb None None] MFA Rules not processed for user `6b98f8d506bc4e6faedb79f2bd87a1ae`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:30.480402 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-66e2f5e8-ff16-4bb7-a7d5-0f28d922eacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:30.547745 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1382/2768] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:30 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:30.559676 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275ab4f-72a8-4554-ba7d-978b57cb7c7c admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:30.561511 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-8275ab4f-72a8-4554-ba7d-978b57cb7c7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:30.614788 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:30.630204 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275ab4f-72a8-4554-ba7d-978b57cb7c7c admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:30.640978 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100b2578-1f1c-49fe-a47b-0991ece6bf8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:30.641619 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b2578-1f1c-49fe-a47b-0991ece6bf8f service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:30.641897 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b2578-1f1c-49fe-a47b-0991ece6bf8f service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:30.642167 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b2578-1f1c-49fe-a47b-0991ece6bf8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:30.652118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100b2578-1f1c-49fe-a47b-0991ece6bf8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:30.653573 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100b2578-1f1c-49fe-a47b-0991ece6bf8f service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:30.662944 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1387/2769] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:00:30 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:00:30.736915 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e2f5e8-ff16-4bb7-a7d5-0f28d922eacb None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:30.747672 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e2f5e8-ff16-4bb7-a7d5-0f28d922eacb None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:30.751260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab06e743-5807-4453-aada-9709323db792 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:30.751869 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06e743-5807-4453-aada-9709323db792 service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:30.752146 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06e743-5807-4453-aada-9709323db792 service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:30.752415 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06e743-5807-4453-aada-9709323db792 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:30.809364 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab06e743-5807-4453-aada-9709323db792 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:30.810803 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab06e743-5807-4453-aada-9709323db792 service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:30.819656 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1383/2770] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:00:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:00:31.084290 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100b2578-1f1c-49fe-a47b-0991ece6bf8f service cinder] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:31.146651 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100b2578-1f1c-49fe-a47b-0991ece6bf8f service cinder] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:31.151970 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb2c1b-bc14-4291-8812-1e85e26eb275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:31.152687 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb2c1b-bc14-4291-8812-1e85e26eb275 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:31.152966 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb2c1b-bc14-4291-8812-1e85e26eb275 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:31.153240 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb2c1b-bc14-4291-8812-1e85e26eb275 admin admin] PATH_INFO: `/v3/users/c9867310516b4fb5800665b97daac11b` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:31.154412 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bb2c1b-bc14-4291-8812-1e85e26eb275 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9867310516b4fb5800665b97daac11b)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:31.156338 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bb2c1b-bc14-4291-8812-1e85e26eb275 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:31.203322 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-34bb2c1b-bc14-4291-8812-1e85e26eb275 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9867310516b4fb5800665b97daac11b'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:31.210620 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-34bb2c1b-bc14-4291-8812-1e85e26eb275 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9867310516b4fb5800665b97daac11b'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:31.218939 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-34bb2c1b-bc14-4291-8812-1e85e26eb275 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9867310516b4fb5800665b97daac11b'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:31.219688 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-34bb2c1b-bc14-4291-8812-1e85e26eb275 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9867310516b4fb5800665b97daac11b'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:31.220377 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-34bb2c1b-bc14-4291-8812-1e85e26eb275 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9867310516b4fb5800665b97daac11b'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:31.237669 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1388/2771] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 23:00:31 2022] DELETE /identity/v3/users/c9867310516b4fb5800665b97daac11b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:31.242171 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f600f63-0f36-414f-acf4-3abb7561dc7c None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:31.242460 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f600f63-0f36-414f-acf4-3abb7561dc7c None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:31.243005 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f600f63-0f36-414f-acf4-3abb7561dc7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:31.263254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-9f600f63-0f36-414f-acf4-3abb7561dc7c None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:31.264496 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-9f600f63-0f36-414f-acf4-3abb7561dc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:31.279854 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=100 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:31.326582 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1384/2772] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:31.336428 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb2c1b-bc14-4291-8812-1e85e26eb275 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:31.337854 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-34bb2c1b-bc14-4291-8812-1e85e26eb275 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:31.395834 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb2c1b-bc14-4291-8812-1e85e26eb275 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:31.400511 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc30997-2d7f-4877-817f-5a15529c1316 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:31.401773 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc30997-2d7f-4877-817f-5a15529c1316 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:31.401975 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc30997-2d7f-4877-817f-5a15529c1316 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:31.402255 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc30997-2d7f-4877-817f-5a15529c1316 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:31.411853 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc30997-2d7f-4877-817f-5a15529c1316 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:31.413487 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc30997-2d7f-4877-817f-5a15529c1316 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:31.427243 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1389/2773] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:00:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:00:31.706645 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f600f63-0f36-414f-acf4-3abb7561dc7c None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:31.708184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-9f600f63-0f36-414f-acf4-3abb7561dc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:31.726284 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f600f63-0f36-414f-acf4-3abb7561dc7c None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:31.732005 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d29783e-1398-4c87-b67f-e1fd62cda2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:31.732646 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29783e-1398-4c87-b67f-e1fd62cda2db admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:31.732908 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29783e-1398-4c87-b67f-e1fd62cda2db admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:31.733173 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29783e-1398-4c87-b67f-e1fd62cda2db admin admin] PATH_INFO: `/v3/projects/2cc07f31b9354c6681368e4216732303` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:31.734485 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d29783e-1398-4c87-b67f-e1fd62cda2db admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cc07f31b9354c6681368e4216732303)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:31.736262 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d29783e-1398-4c87-b67f-e1fd62cda2db admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:31.783575 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-1d29783e-1398-4c87-b67f-e1fd62cda2db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cc07f31b9354c6681368e4216732303', 'request_id': 'req-1d29783e-1398-4c87-b67f-e1fd62cda2db'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:31.792308 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-1d29783e-1398-4c87-b67f-e1fd62cda2db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cc07f31b9354c6681368e4216732303', 'request_id': 'req-1d29783e-1398-4c87-b67f-e1fd62cda2db'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:31.797980 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-1d29783e-1398-4c87-b67f-e1fd62cda2db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cc07f31b9354c6681368e4216732303', 'request_id': 'req-1d29783e-1398-4c87-b67f-e1fd62cda2db'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:31.810257 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-1d29783e-1398-4c87-b67f-e1fd62cda2db admin admin] The token cache is being invalidate because project 2cc07f31b9354c6681368e4216732303 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:00:31.810570 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-1d29783e-1398-4c87-b67f-e1fd62cda2db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:31.811129 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-1d29783e-1398-4c87-b67f-e1fd62cda2db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:31.812743 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1385/2774] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 23:00:31 2022] DELETE /identity/v3/projects/2cc07f31b9354c6681368e4216732303 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:31.816574 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc30997-2d7f-4877-817f-5a15529c1316 service neutron] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:31.827264 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc30997-2d7f-4877-817f-5a15529c1316 service neutron] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:31.831047 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:31.884392 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b66d2cc-2d0d-4dfb-97f6-1bd53ae35f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:31.885034 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66d2cc-2d0d-4dfb-97f6-1bd53ae35f67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:31.886023 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66d2cc-2d0d-4dfb-97f6-1bd53ae35f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:31.886299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66d2cc-2d0d-4dfb-97f6-1bd53ae35f67 admin admin] PATH_INFO: `/v3/users/6b98f8d506bc4e6faedb79f2bd87a1ae` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:31.887432 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b66d2cc-2d0d-4dfb-97f6-1bd53ae35f67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b98f8d506bc4e6faedb79f2bd87a1ae)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:31.889376 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b66d2cc-2d0d-4dfb-97f6-1bd53ae35f67 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:31.937605 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-4b66d2cc-2d0d-4dfb-97f6-1bd53ae35f67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b98f8d506bc4e6faedb79f2bd87a1ae'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:31.946449 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-4b66d2cc-2d0d-4dfb-97f6-1bd53ae35f67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b98f8d506bc4e6faedb79f2bd87a1ae'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:31.954818 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-4b66d2cc-2d0d-4dfb-97f6-1bd53ae35f67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b98f8d506bc4e6faedb79f2bd87a1ae'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:31.955492 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-4b66d2cc-2d0d-4dfb-97f6-1bd53ae35f67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b98f8d506bc4e6faedb79f2bd87a1ae'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:31.956100 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-4b66d2cc-2d0d-4dfb-97f6-1bd53ae35f67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b98f8d506bc4e6faedb79f2bd87a1ae'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:31.973843 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1390/2775] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 23:00:31 2022] DELETE /identity/v3/users/6b98f8d506bc4e6faedb79f2bd87a1ae => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:32.249821 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d29783e-1398-4c87-b67f-e1fd62cda2db admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:32.251631 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-1d29783e-1398-4c87-b67f-e1fd62cda2db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:32.267425 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d29783e-1398-4c87-b67f-e1fd62cda2db admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:32.326576 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a269e-fc6b-4421-8f7f-334ce80a03f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:32.327207 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a269e-fc6b-4421-8f7f-334ce80a03f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:32.327487 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a269e-fc6b-4421-8f7f-334ce80a03f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:32.327757 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a269e-fc6b-4421-8f7f-334ce80a03f3 admin admin] PATH_INFO: `/v3/projects/eb8b7858524e42b08d6f1636bbb7f64c` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:32.328857 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a269e-fc6b-4421-8f7f-334ce80a03f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb8b7858524e42b08d6f1636bbb7f64c)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:32.330790 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a269e-fc6b-4421-8f7f-334ce80a03f3 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:32.373438 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:32.377454 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-5a0a269e-fc6b-4421-8f7f-334ce80a03f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb8b7858524e42b08d6f1636bbb7f64c', 'request_id': 'req-5a0a269e-fc6b-4421-8f7f-334ce80a03f3'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:32.384711 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-5a0a269e-fc6b-4421-8f7f-334ce80a03f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb8b7858524e42b08d6f1636bbb7f64c', 'request_id': 'req-5a0a269e-fc6b-4421-8f7f-334ce80a03f3'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:32.390417 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-5a0a269e-fc6b-4421-8f7f-334ce80a03f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb8b7858524e42b08d6f1636bbb7f64c', 'request_id': 'req-5a0a269e-fc6b-4421-8f7f-334ce80a03f3'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:32.401842 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-5a0a269e-fc6b-4421-8f7f-334ce80a03f3 admin admin] The token cache is being invalidate because project eb8b7858524e42b08d6f1636bbb7f64c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:00:32.402107 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-5a0a269e-fc6b-4421-8f7f-334ce80a03f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:32.402764 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-5a0a269e-fc6b-4421-8f7f-334ce80a03f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:32.404224 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1386/2776] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 23:00:32 2022] DELETE /identity/v3/projects/eb8b7858524e42b08d6f1636bbb7f64c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:32.409291 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79adea97-15ea-4392-9f9b-c7fa77839706 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:32.409625 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79adea97-15ea-4392-9f9b-c7fa77839706 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:32.409901 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79adea97-15ea-4392-9f9b-c7fa77839706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:32.430541 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-79adea97-15ea-4392-9f9b-c7fa77839706 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:32.431769 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-79adea97-15ea-4392-9f9b-c7fa77839706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:32.490656 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1391/2777] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:32.499344 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225dafd6-ddeb-4cb8-86c3-5ccd63850ea1 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:32.499659 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225dafd6-ddeb-4cb8-86c3-5ccd63850ea1 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:32.499937 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225dafd6-ddeb-4cb8-86c3-5ccd63850ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:32.520676 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-225dafd6-ddeb-4cb8-86c3-5ccd63850ea1 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:32.522004 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-225dafd6-ddeb-4cb8-86c3-5ccd63850ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:32.530150 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1387/2778] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:32.534366 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79adea97-15ea-4392-9f9b-c7fa77839706 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:32.544708 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79adea97-15ea-4392-9f9b-c7fa77839706 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:32.549244 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ffeb8-1f0d-48b7-8add-65cdcb58692c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:32.549871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ffeb8-1f0d-48b7-8add-65cdcb58692c admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:32.550168 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ffeb8-1f0d-48b7-8add-65cdcb58692c admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:32.550455 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ffeb8-1f0d-48b7-8add-65cdcb58692c admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:32.551100 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8ffeb8-1f0d-48b7-8add-65cdcb58692c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:00:32.551400 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8ffeb8-1f0d-48b7-8add-65cdcb58692c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:32.552996 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8ffeb8-1f0d-48b7-8add-65cdcb58692c admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:32.565175 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1392/2779] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 23:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 23:00:32.569537 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225dafd6-ddeb-4cb8-86c3-5ccd63850ea1 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:32.578855 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225dafd6-ddeb-4cb8-86c3-5ccd63850ea1 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:32.583460 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e321e-2d42-439c-a03c-e64d057f70d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:32.584045 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e321e-2d42-439c-a03c-e64d057f70d6 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:32.584329 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e321e-2d42-439c-a03c-e64d057f70d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:32.584582 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e321e-2d42-439c-a03c-e64d057f70d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:32.585211 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e321e-2d42-439c-a03c-e64d057f70d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:32.587217 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e321e-2d42-439c-a03c-e64d057f70d6 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:32.614509 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1388/2780] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 23:00:32 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:00:32.618863 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ffeb8-1f0d-48b7-8add-65cdcb58692c admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:32.629512 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ffeb8-1f0d-48b7-8add-65cdcb58692c admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:32.638884 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:32.690327 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0aa886-5ee7-4b4f-895a-bb4224d8ce96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:32.690940 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0aa886-5ee7-4b4f-895a-bb4224d8ce96 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:32.691220 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0aa886-5ee7-4b4f-895a-bb4224d8ce96 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:32.691494 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0aa886-5ee7-4b4f-895a-bb4224d8ce96 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:32.692112 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0aa886-5ee7-4b4f-895a-bb4224d8ce96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:32.694091 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0aa886-5ee7-4b4f-895a-bb4224d8ce96 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:32.714895 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1393/2781] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 23:00:32 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:00:32.719079 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e321e-2d42-439c-a03c-e64d057f70d6 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:32.730853 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e321e-2d42-439c-a03c-e64d057f70d6 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:32.735533 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65001a51-013d-46c5-ba9f-08da10774fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:32.736132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65001a51-013d-46c5-ba9f-08da10774fda admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:32.736411 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65001a51-013d-46c5-ba9f-08da10774fda admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:32.736685 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65001a51-013d-46c5-ba9f-08da10774fda admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:32.737355 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65001a51-013d-46c5-ba9f-08da10774fda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:32.738969 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65001a51-013d-46c5-ba9f-08da10774fda admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:32.750571 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1389/2782] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:00:32.754282 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0aa886-5ee7-4b4f-895a-bb4224d8ce96 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:32.764321 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0aa886-5ee7-4b4f-895a-bb4224d8ce96 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:32.768904 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600471d-2529-47bf-81b4-ea3e46eb5222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:32.769560 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600471d-2529-47bf-81b4-ea3e46eb5222 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:32.769839 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600471d-2529-47bf-81b4-ea3e46eb5222 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:32.770108 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600471d-2529-47bf-81b4-ea3e46eb5222 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:32.770740 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600471d-2529-47bf-81b4-ea3e46eb5222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:32.772328 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600471d-2529-47bf-81b4-ea3e46eb5222 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:32.782723 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1394/2783] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:00:32.786970 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65001a51-013d-46c5-ba9f-08da10774fda admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:32.797922 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65001a51-013d-46c5-ba9f-08da10774fda admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:32.802590 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc34fe61-101c-4fdd-8f1a-808e570b48a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:32.803238 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34fe61-101c-4fdd-8f1a-808e570b48a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:32.803520 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34fe61-101c-4fdd-8f1a-808e570b48a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:32.803795 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34fe61-101c-4fdd-8f1a-808e570b48a1 admin admin] PATH_INFO: `/v3/projects/75b7f3845c004c02ac9964c459eb11b7/users/8d6b5d8f849e42138b0925d12a470c21/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:32.819844 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc34fe61-101c-4fdd-8f1a-808e570b48a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75b7f3845c004c02ac9964c459eb11b7, user_id=8d6b5d8f849e42138b0925d12a470c21, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:32.822546 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc34fe61-101c-4fdd-8f1a-808e570b48a1 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:32.835456 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1390/2784] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 23:00:32 2022] PUT /identity/v3/projects/75b7f3845c004c02ac9964c459eb11b7/users/8d6b5d8f849e42138b0925d12a470c21/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:32.846197 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d27a5d-a256-4193-b071-d9eec5d52b56 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:32.846727 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d27a5d-a256-4193-b071-d9eec5d52b56 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:32.846832 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d27a5d-a256-4193-b071-d9eec5d52b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:32.875615 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-f6d27a5d-a256-4193-b071-d9eec5d52b56 None None] MFA Rules not processed for user `8d6b5d8f849e42138b0925d12a470c21`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:32.876836 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-f6d27a5d-a256-4193-b071-d9eec5d52b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:32.939173 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1395/2785] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:32.943497 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc34fe61-101c-4fdd-8f1a-808e570b48a1 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:33.005692 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc34fe61-101c-4fdd-8f1a-808e570b48a1 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:33.010769 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c78ec10-8495-42a2-91bf-0d69dff29896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:33.011572 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78ec10-8495-42a2-91bf-0d69dff29896 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:33.011854 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78ec10-8495-42a2-91bf-0d69dff29896 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:33.012127 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78ec10-8495-42a2-91bf-0d69dff29896 admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:33.012755 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c78ec10-8495-42a2-91bf-0d69dff29896 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:33.014707 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c78ec10-8495-42a2-91bf-0d69dff29896 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:33.043299 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1391/2786] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 23:00:32 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:00:33.047402 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d27a5d-a256-4193-b071-d9eec5d52b56 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:33.058767 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d27a5d-a256-4193-b071-d9eec5d52b56 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:33.113412 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94efae51-8eda-4776-b450-381a80e73994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:33.114022 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94efae51-8eda-4776-b450-381a80e73994 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:33.114308 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94efae51-8eda-4776-b450-381a80e73994 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:33.114585 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94efae51-8eda-4776-b450-381a80e73994 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:33.115212 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94efae51-8eda-4776-b450-381a80e73994 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:33.117142 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94efae51-8eda-4776-b450-381a80e73994 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:33.140777 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1396/2787] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 23:00:33 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:00:33.145032 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c78ec10-8495-42a2-91bf-0d69dff29896 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:33.155851 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c78ec10-8495-42a2-91bf-0d69dff29896 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:33.160501 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad009cd-901c-48f2-8a8b-9ac5e65968c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:33.161101 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad009cd-901c-48f2-8a8b-9ac5e65968c7 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:33.161422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad009cd-901c-48f2-8a8b-9ac5e65968c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:33.161697 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad009cd-901c-48f2-8a8b-9ac5e65968c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:33.162354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad009cd-901c-48f2-8a8b-9ac5e65968c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:33.163944 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad009cd-901c-48f2-8a8b-9ac5e65968c7 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:33.175651 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1392/2788] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:00:33.179778 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94efae51-8eda-4776-b450-381a80e73994 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:33.190232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94efae51-8eda-4776-b450-381a80e73994 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:33.194803 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ea179-3cdd-4655-9a6e-737db51e0058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:33.195396 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ea179-3cdd-4655-9a6e-737db51e0058 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:33.195677 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ea179-3cdd-4655-9a6e-737db51e0058 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:33.195951 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ea179-3cdd-4655-9a6e-737db51e0058 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:33.196583 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ea179-3cdd-4655-9a6e-737db51e0058 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:33.198345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ea179-3cdd-4655-9a6e-737db51e0058 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:33.203565 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:33.209725 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1397/2789] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:00:33.214260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad009cd-901c-48f2-8a8b-9ac5e65968c7 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:33.225719 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad009cd-901c-48f2-8a8b-9ac5e65968c7 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:33.230481 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244cb4d-0740-4fa3-8b46-66467d4ce629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:33.231130 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244cb4d-0740-4fa3-8b46-66467d4ce629 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:33.231409 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244cb4d-0740-4fa3-8b46-66467d4ce629 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:33.231685 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244cb4d-0740-4fa3-8b46-66467d4ce629 admin admin] PATH_INFO: `/v3/projects/5eff7dc5b2a9416299cc33fe17258fd8/users/c5325aeed76e40c098cd2b3d19781fa2/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:33.247005 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244cb4d-0740-4fa3-8b46-66467d4ce629 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eff7dc5b2a9416299cc33fe17258fd8, user_id=c5325aeed76e40c098cd2b3d19781fa2, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:33.249695 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244cb4d-0740-4fa3-8b46-66467d4ce629 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:33.263086 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1393/2790] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 23:00:33 2022] PUT /identity/v3/projects/5eff7dc5b2a9416299cc33fe17258fd8/users/c5325aeed76e40c098cd2b3d19781fa2/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:33.274473 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d802d70-c2a0-4435-a60b-98c3079a977c None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:33.274760 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d802d70-c2a0-4435-a60b-98c3079a977c None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:33.275034 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d802d70-c2a0-4435-a60b-98c3079a977c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:33.303105 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-3d802d70-c2a0-4435-a60b-98c3079a977c None None] MFA Rules not processed for user `c5325aeed76e40c098cd2b3d19781fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:33.304313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-3d802d70-c2a0-4435-a60b-98c3079a977c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:33.367993 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1398/2791] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:33.374131 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244cb4d-0740-4fa3-8b46-66467d4ce629 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:33.394159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:33.439575 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244cb4d-0740-4fa3-8b46-66467d4ce629 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:33.444178 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff4b18-77bb-4edd-8f16-fe6c1a6bc4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:33.444785 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff4b18-77bb-4edd-8f16-fe6c1a6bc4ec admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:33.445062 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff4b18-77bb-4edd-8f16-fe6c1a6bc4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:33.445362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff4b18-77bb-4edd-8f16-fe6c1a6bc4ec admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:33.445987 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff4b18-77bb-4edd-8f16-fe6c1a6bc4ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:33.448725 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff4b18-77bb-4edd-8f16-fe6c1a6bc4ec admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:33.475074 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1394/2792] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 23:00:33 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:00:33.479615 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d802d70-c2a0-4435-a60b-98c3079a977c None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:33.491083 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d802d70-c2a0-4435-a60b-98c3079a977c None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:33.551797 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6dd59-bd11-4108-90a4-880af22f38b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:33.552313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6dd59-bd11-4108-90a4-880af22f38b6 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:33.552595 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6dd59-bd11-4108-90a4-880af22f38b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:33.552885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6dd59-bd11-4108-90a4-880af22f38b6 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:33.553554 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c6dd59-bd11-4108-90a4-880af22f38b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:33.555479 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c6dd59-bd11-4108-90a4-880af22f38b6 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:33.575825 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1399/2793] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 23:00:33 2022] POST /identity/v3/users => generated 295 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:00:33.579857 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff4b18-77bb-4edd-8f16-fe6c1a6bc4ec admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:33.591029 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff4b18-77bb-4edd-8f16-fe6c1a6bc4ec admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:33.596073 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70536506-8cc4-499e-8892-668c0dcddeb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:33.596679 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70536506-8cc4-499e-8892-668c0dcddeb4 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:33.596959 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70536506-8cc4-499e-8892-668c0dcddeb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:33.597232 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70536506-8cc4-499e-8892-668c0dcddeb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:33.597911 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70536506-8cc4-499e-8892-668c0dcddeb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:33.599511 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70536506-8cc4-499e-8892-668c0dcddeb4 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:33.611314 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1395/2794] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:00:33.615530 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6dd59-bd11-4108-90a4-880af22f38b6 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:33.626506 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6dd59-bd11-4108-90a4-880af22f38b6 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:33.631269 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5733dd-9ba5-422a-ab36-758fcb1989a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:33.632094 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5733dd-9ba5-422a-ab36-758fcb1989a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:33.632383 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5733dd-9ba5-422a-ab36-758fcb1989a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:33.632658 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5733dd-9ba5-422a-ab36-758fcb1989a1 admin admin] PATH_INFO: `/v3/projects/a8c99bbf674e4578b985fc7e68cd08c8/users/6e03e4f8e2f34867bed146ed69ce94ab/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:33.647971 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5733dd-9ba5-422a-ab36-758fcb1989a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8c99bbf674e4578b985fc7e68cd08c8, user_id=6e03e4f8e2f34867bed146ed69ce94ab, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:33.650889 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5733dd-9ba5-422a-ab36-758fcb1989a1 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:33.664380 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1400/2795] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 23:00:33 2022] PUT /identity/v3/projects/a8c99bbf674e4578b985fc7e68cd08c8/users/6e03e4f8e2f34867bed146ed69ce94ab/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:33.669180 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70536506-8cc4-499e-8892-668c0dcddeb4 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:33.679288 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70536506-8cc4-499e-8892-668c0dcddeb4 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:33.740619 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a83ddb2-7e19-4d74-bc72-70b315564184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:33.741265 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83ddb2-7e19-4d74-bc72-70b315564184 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:33.741690 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83ddb2-7e19-4d74-bc72-70b315564184 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:33.741893 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83ddb2-7e19-4d74-bc72-70b315564184 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:33.742862 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a83ddb2-7e19-4d74-bc72-70b315564184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:33.745104 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a83ddb2-7e19-4d74-bc72-70b315564184 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:33.756141 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1396/2796] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:00:33.760661 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5733dd-9ba5-422a-ab36-758fcb1989a1 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:33.771867 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5733dd-9ba5-422a-ab36-758fcb1989a1 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:33.776537 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15081936-1ad4-4887-ab2a-9d32a9cae072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:33.777196 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15081936-1ad4-4887-ab2a-9d32a9cae072 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:33.777509 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15081936-1ad4-4887-ab2a-9d32a9cae072 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:33.777786 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15081936-1ad4-4887-ab2a-9d32a9cae072 admin admin] PATH_INFO: `/v3/projects/a8c99bbf674e4578b985fc7e68cd08c8/users/6e03e4f8e2f34867bed146ed69ce94ab/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:33.779758 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15081936-1ad4-4887-ab2a-9d32a9cae072 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8c99bbf674e4578b985fc7e68cd08c8, user_id=6e03e4f8e2f34867bed146ed69ce94ab, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:33.782442 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15081936-1ad4-4887-ab2a-9d32a9cae072 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:33.794903 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1401/2797] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 23:00:33 2022] PUT /identity/v3/projects/a8c99bbf674e4578b985fc7e68cd08c8/users/6e03e4f8e2f34867bed146ed69ce94ab/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:00:33.806393 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb269bf-d158-4148-a65f-173222dfc643 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:33.806520 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb269bf-d158-4148-a65f-173222dfc643 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:33.806993 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb269bf-d158-4148-a65f-173222dfc643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:33.835951 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-9cb269bf-d158-4148-a65f-173222dfc643 None None] MFA Rules not processed for user `6e03e4f8e2f34867bed146ed69ce94ab`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:33.837349 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-9cb269bf-d158-4148-a65f-173222dfc643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:33.910817 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1397/2798] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:33 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:33.923988 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15081936-1ad4-4887-ab2a-9d32a9cae072 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:33.925840 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-15081936-1ad4-4887-ab2a-9d32a9cae072 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:33.981684 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:33.995244 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15081936-1ad4-4887-ab2a-9d32a9cae072 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:33.999133 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d63a4-2e17-4d81-b714-4bd066c07334 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:33.999747 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d63a4-2e17-4d81-b714-4bd066c07334 service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:34.000031 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d63a4-2e17-4d81-b714-4bd066c07334 service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:34.000308 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d63a4-2e17-4d81-b714-4bd066c07334 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:34.060682 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36d63a4-2e17-4d81-b714-4bd066c07334 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:34.062124 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36d63a4-2e17-4d81-b714-4bd066c07334 service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:34.071937 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1402/2799] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:00:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:00:35.611665 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb269bf-d158-4148-a65f-173222dfc643 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:35.622013 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb269bf-d158-4148-a65f-173222dfc643 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:35.625823 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95541e81-79e4-4b92-95f8-c2bdc3d09fe1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:35.626426 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95541e81-79e4-4b92-95f8-c2bdc3d09fe1 service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:35.626707 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95541e81-79e4-4b92-95f8-c2bdc3d09fe1 service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:35.626983 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95541e81-79e4-4b92-95f8-c2bdc3d09fe1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:35.636670 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:35.688369 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95541e81-79e4-4b92-95f8-c2bdc3d09fe1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:35.689852 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95541e81-79e4-4b92-95f8-c2bdc3d09fe1 service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:35.699307 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1398/2800] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:00:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:00:36.235047 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d63a4-2e17-4d81-b714-4bd066c07334 service cinder] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:36.245044 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d63a4-2e17-4d81-b714-4bd066c07334 service cinder] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:36.248826 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c62007-ac0d-4f5b-be5a-9bb859882374 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:36.249487 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c62007-ac0d-4f5b-be5a-9bb859882374 service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:36.249768 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c62007-ac0d-4f5b-be5a-9bb859882374 service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:36.250042 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c62007-ac0d-4f5b-be5a-9bb859882374 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:36.259134 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c62007-ac0d-4f5b-be5a-9bb859882374 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:36.261133 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c62007-ac0d-4f5b-be5a-9bb859882374 service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:36.270610 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1403/2801] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:00:36 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:00:36.694071 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95541e81-79e4-4b92-95f8-c2bdc3d09fe1 service cinder] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:36.695484 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-95541e81-79e4-4b92-95f8-c2bdc3d09fe1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:36.762162 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95541e81-79e4-4b92-95f8-c2bdc3d09fe1 service cinder] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:36.766599 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810beccd-ba22-45a7-9502-672a3cc3708b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:36.767213 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810beccd-ba22-45a7-9502-672a3cc3708b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:36.767490 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810beccd-ba22-45a7-9502-672a3cc3708b admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:36.767757 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810beccd-ba22-45a7-9502-672a3cc3708b admin admin] PATH_INFO: `/v3/users/a6e64cdb87404c51b1b65d66207ba7f5` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:36.768868 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810beccd-ba22-45a7-9502-672a3cc3708b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6e64cdb87404c51b1b65d66207ba7f5)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:36.770804 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810beccd-ba22-45a7-9502-672a3cc3708b admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:36.816315 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-810beccd-ba22-45a7-9502-672a3cc3708b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6e64cdb87404c51b1b65d66207ba7f5'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:36.823177 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-810beccd-ba22-45a7-9502-672a3cc3708b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6e64cdb87404c51b1b65d66207ba7f5'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:36.834743 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-810beccd-ba22-45a7-9502-672a3cc3708b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6e64cdb87404c51b1b65d66207ba7f5'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:36.835419 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-810beccd-ba22-45a7-9502-672a3cc3708b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6e64cdb87404c51b1b65d66207ba7f5'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:36.836018 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-810beccd-ba22-45a7-9502-672a3cc3708b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6e64cdb87404c51b1b65d66207ba7f5'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:36.849260 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1399/2802] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 23:00:36 2022] DELETE /identity/v3/users/a6e64cdb87404c51b1b65d66207ba7f5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:36.854120 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6254629-030a-4ebc-be3c-533a8df8034f None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:36.854352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6254629-030a-4ebc-be3c-533a8df8034f None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:36.854649 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6254629-030a-4ebc-be3c-533a8df8034f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:36.874902 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-d6254629-030a-4ebc-be3c-533a8df8034f None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:36.876157 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-d6254629-030a-4ebc-be3c-533a8df8034f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:36.935177 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1404/2803] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:36.944920 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810beccd-ba22-45a7-9502-672a3cc3708b admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:36.946348 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-810beccd-ba22-45a7-9502-672a3cc3708b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:36.981493 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:37.006135 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810beccd-ba22-45a7-9502-672a3cc3708b admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:37.010321 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa37ef-7d6b-44da-964e-dee9477a2ae8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:37.010914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa37ef-7d6b-44da-964e-dee9477a2ae8 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:37.011189 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa37ef-7d6b-44da-964e-dee9477a2ae8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:37.011455 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa37ef-7d6b-44da-964e-dee9477a2ae8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:37.020247 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aa37ef-7d6b-44da-964e-dee9477a2ae8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:37.021688 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aa37ef-7d6b-44da-964e-dee9477a2ae8 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:37.031222 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1400/2804] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:00:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:00:37.312730 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6254629-030a-4ebc-be3c-533a8df8034f None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:37.314268 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-d6254629-030a-4ebc-be3c-533a8df8034f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:37.328447 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6254629-030a-4ebc-be3c-533a8df8034f None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:37.333142 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d23cf-6690-460b-a7be-252137a883ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:37.333836 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d23cf-6690-460b-a7be-252137a883ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:37.334120 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d23cf-6690-460b-a7be-252137a883ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:37.334801 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d23cf-6690-460b-a7be-252137a883ac admin admin] PATH_INFO: `/v3/projects/032c383b27474bb68698557ffa79f965` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:37.335719 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d23cf-6690-460b-a7be-252137a883ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=032c383b27474bb68698557ffa79f965)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:37.338513 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d23cf-6690-460b-a7be-252137a883ac admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:37.388242 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-da4d23cf-6690-460b-a7be-252137a883ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '032c383b27474bb68698557ffa79f965', 'request_id': 'req-da4d23cf-6690-460b-a7be-252137a883ac'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:37.393863 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-da4d23cf-6690-460b-a7be-252137a883ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '032c383b27474bb68698557ffa79f965', 'request_id': 'req-da4d23cf-6690-460b-a7be-252137a883ac'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:37.405326 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-da4d23cf-6690-460b-a7be-252137a883ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '032c383b27474bb68698557ffa79f965', 'request_id': 'req-da4d23cf-6690-460b-a7be-252137a883ac'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:37.418345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-da4d23cf-6690-460b-a7be-252137a883ac admin admin] The token cache is being invalidate because project 032c383b27474bb68698557ffa79f965 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:00:37.418523 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-da4d23cf-6690-460b-a7be-252137a883ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:37.419065 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-da4d23cf-6690-460b-a7be-252137a883ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:37.420555 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1405/2805] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 23:00:37 2022] DELETE /identity/v3/projects/032c383b27474bb68698557ffa79f965 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:37.424550 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa37ef-7d6b-44da-964e-dee9477a2ae8 service neutron] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:37.435143 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa37ef-7d6b-44da-964e-dee9477a2ae8 service neutron] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:37.493363 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b60ee-4ebd-43ed-a7cf-4266027170d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:37.494050 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b60ee-4ebd-43ed-a7cf-4266027170d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:37.494325 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b60ee-4ebd-43ed-a7cf-4266027170d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:37.494735 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b60ee-4ebd-43ed-a7cf-4266027170d8 admin admin] PATH_INFO: `/v3/users/1fb6ab963afa4c1ca14dfc17278cf67c` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:37.495881 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b60ee-4ebd-43ed-a7cf-4266027170d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fb6ab963afa4c1ca14dfc17278cf67c)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:37.497816 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b60ee-4ebd-43ed-a7cf-4266027170d8 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:37.547404 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-553b60ee-4ebd-43ed-a7cf-4266027170d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fb6ab963afa4c1ca14dfc17278cf67c'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:37.554785 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-553b60ee-4ebd-43ed-a7cf-4266027170d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fb6ab963afa4c1ca14dfc17278cf67c'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:37.569349 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-553b60ee-4ebd-43ed-a7cf-4266027170d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fb6ab963afa4c1ca14dfc17278cf67c'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:37.570078 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-553b60ee-4ebd-43ed-a7cf-4266027170d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fb6ab963afa4c1ca14dfc17278cf67c'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:37.571164 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-553b60ee-4ebd-43ed-a7cf-4266027170d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fb6ab963afa4c1ca14dfc17278cf67c'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:37.591346 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1401/2806] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 23:00:37 2022] DELETE /identity/v3/users/1fb6ab963afa4c1ca14dfc17278cf67c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:37.881460 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d23cf-6690-460b-a7be-252137a883ac admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:37.882965 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-da4d23cf-6690-460b-a7be-252137a883ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:37.896842 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d23cf-6690-460b-a7be-252137a883ac admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:37.901148 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:37.959577 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c5353a-3745-4e3a-8671-8280ee247b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:37.960232 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5353a-3745-4e3a-8671-8280ee247b9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:37.960497 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5353a-3745-4e3a-8671-8280ee247b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:37.960774 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5353a-3745-4e3a-8671-8280ee247b9f admin admin] PATH_INFO: `/v3/projects/456da5b5bb244dbb931d78f4b28f2d6a` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:37.961924 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c5353a-3745-4e3a-8671-8280ee247b9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=456da5b5bb244dbb931d78f4b28f2d6a)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:37.963916 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c5353a-3745-4e3a-8671-8280ee247b9f admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:38.009677 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-b0c5353a-3745-4e3a-8671-8280ee247b9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '456da5b5bb244dbb931d78f4b28f2d6a', 'request_id': 'req-b0c5353a-3745-4e3a-8671-8280ee247b9f'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:38.015034 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-b0c5353a-3745-4e3a-8671-8280ee247b9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '456da5b5bb244dbb931d78f4b28f2d6a', 'request_id': 'req-b0c5353a-3745-4e3a-8671-8280ee247b9f'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:38.022233 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-b0c5353a-3745-4e3a-8671-8280ee247b9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '456da5b5bb244dbb931d78f4b28f2d6a', 'request_id': 'req-b0c5353a-3745-4e3a-8671-8280ee247b9f'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:38.033078 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-b0c5353a-3745-4e3a-8671-8280ee247b9f admin admin] The token cache is being invalidate because project 456da5b5bb244dbb931d78f4b28f2d6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:00:38.033714 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-b0c5353a-3745-4e3a-8671-8280ee247b9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:38.034112 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-b0c5353a-3745-4e3a-8671-8280ee247b9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:38.035588 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1406/2807] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 23:00:37 2022] DELETE /identity/v3/projects/456da5b5bb244dbb931d78f4b28f2d6a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:38.042094 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36394351-ec69-41c7-a158-c807aaa2106b None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:38.042586 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36394351-ec69-41c7-a158-c807aaa2106b None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:38.042690 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36394351-ec69-41c7-a158-c807aaa2106b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:38.062405 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-36394351-ec69-41c7-a158-c807aaa2106b None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:38.063611 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-36394351-ec69-41c7-a158-c807aaa2106b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:38.110236 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:38.127446 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1402/2808] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:38.135562 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c6cac-ddd9-4cf1-8333-d631cf236120 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:38.135780 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c6cac-ddd9-4cf1-8333-d631cf236120 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:38.136057 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c6cac-ddd9-4cf1-8333-d631cf236120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:38.155408 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-9e1c6cac-ddd9-4cf1-8333-d631cf236120 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:38.156623 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-9e1c6cac-ddd9-4cf1-8333-d631cf236120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:38.164516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1407/2809] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:38.169451 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36394351-ec69-41c7-a158-c807aaa2106b None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:38.179095 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36394351-ec69-41c7-a158-c807aaa2106b None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:38.184217 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d09cd2-11c4-41b7-a31d-f5ec71f1a98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:38.184812 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d09cd2-11c4-41b7-a31d-f5ec71f1a98d admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:38.185088 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d09cd2-11c4-41b7-a31d-f5ec71f1a98d admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:38.185382 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d09cd2-11c4-41b7-a31d-f5ec71f1a98d admin admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:38.186019 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d09cd2-11c4-41b7-a31d-f5ec71f1a98d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:00:38.186317 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d09cd2-11c4-41b7-a31d-f5ec71f1a98d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:38.187897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d09cd2-11c4-41b7-a31d-f5ec71f1a98d admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:38.200146 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1403/2810] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 23:00:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:00:38.204516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1c6cac-ddd9-4cf1-8333-d631cf236120 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:38.214457 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1c6cac-ddd9-4cf1-8333-d631cf236120 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:38.219116 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11c282-b9b1-4d9f-bfd7-356168101ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:38.219720 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11c282-b9b1-4d9f-bfd7-356168101ab5 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:38.219999 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11c282-b9b1-4d9f-bfd7-356168101ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:38.220271 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11c282-b9b1-4d9f-bfd7-356168101ab5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:38.220889 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb11c282-b9b1-4d9f-bfd7-356168101ab5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:38.222848 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb11c282-b9b1-4d9f-bfd7-356168101ab5 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:38.250164 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1408/2811] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 23:00:38 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:00:38.254402 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d09cd2-11c4-41b7-a31d-f5ec71f1a98d admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:38.265052 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d09cd2-11c4-41b7-a31d-f5ec71f1a98d admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:38.323512 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c84707-9b3b-462c-97d9-a61ffa192bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:38.324414 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c84707-9b3b-462c-97d9-a61ffa192bc9 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:38.324697 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c84707-9b3b-462c-97d9-a61ffa192bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:38.324975 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c84707-9b3b-462c-97d9-a61ffa192bc9 admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:38.325685 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c84707-9b3b-462c-97d9-a61ffa192bc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:38.327859 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c84707-9b3b-462c-97d9-a61ffa192bc9 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:38.349397 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1404/2812] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 23:00:38 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:00:38.353418 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11c282-b9b1-4d9f-bfd7-356168101ab5 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:38.363628 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11c282-b9b1-4d9f-bfd7-356168101ab5 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:38.368482 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83cb8d-f791-45e0-96f0-320418dd3beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:38.369815 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83cb8d-f791-45e0-96f0-320418dd3beb admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:38.370090 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83cb8d-f791-45e0-96f0-320418dd3beb admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:38.370370 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83cb8d-f791-45e0-96f0-320418dd3beb admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:38.370986 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a83cb8d-f791-45e0-96f0-320418dd3beb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:38.372629 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a83cb8d-f791-45e0-96f0-320418dd3beb admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:38.382399 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1409/2813] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:00:38.387132 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c84707-9b3b-462c-97d9-a61ffa192bc9 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:38.397072 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c84707-9b3b-462c-97d9-a61ffa192bc9 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:38.402053 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f925a78-4e77-4060-81ed-5763014e2efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:38.402650 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f925a78-4e77-4060-81ed-5763014e2efa admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:38.402930 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f925a78-4e77-4060-81ed-5763014e2efa admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:38.403202 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f925a78-4e77-4060-81ed-5763014e2efa admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:38.403832 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f925a78-4e77-4060-81ed-5763014e2efa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:38.405472 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f925a78-4e77-4060-81ed-5763014e2efa admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:38.416231 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1405/2814] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:00:38.420251 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83cb8d-f791-45e0-96f0-320418dd3beb admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:38.434379 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83cb8d-f791-45e0-96f0-320418dd3beb admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:38.439174 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297a635-4f84-4d2e-8d89-c7ac17a4d303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:38.439834 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297a635-4f84-4d2e-8d89-c7ac17a4d303 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:38.440118 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297a635-4f84-4d2e-8d89-c7ac17a4d303 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:38.440399 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297a635-4f84-4d2e-8d89-c7ac17a4d303 admin admin] PATH_INFO: `/v3/projects/4306706a5b3e49bdad9c75b899b470d0/users/36bed85f60024b539767dc72a3f68eab/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:38.455440 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297a635-4f84-4d2e-8d89-c7ac17a4d303 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4306706a5b3e49bdad9c75b899b470d0, user_id=36bed85f60024b539767dc72a3f68eab, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:38.458204 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297a635-4f84-4d2e-8d89-c7ac17a4d303 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:38.472299 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1410/2815] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 23:00:38 2022] PUT /identity/v3/projects/4306706a5b3e49bdad9c75b899b470d0/users/36bed85f60024b539767dc72a3f68eab/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:38.484535 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd3153c-0256-4289-80fb-83b0aa0eaf9e None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:38.484828 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd3153c-0256-4289-80fb-83b0aa0eaf9e None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:38.485134 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd3153c-0256-4289-80fb-83b0aa0eaf9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:38.512231 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-4dd3153c-0256-4289-80fb-83b0aa0eaf9e None None] MFA Rules not processed for user `36bed85f60024b539767dc72a3f68eab`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:38.513527 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-4dd3153c-0256-4289-80fb-83b0aa0eaf9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:38.576694 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1406/2816] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:38.588954 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297a635-4f84-4d2e-8d89-c7ac17a4d303 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:38.590707 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-f297a635-4f84-4d2e-8d89-c7ac17a4d303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:38.628405 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:38.656708 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297a635-4f84-4d2e-8d89-c7ac17a4d303 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:38.660302 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f893e-a9b3-4be0-989a-51c6330310de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:38.660908 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f893e-a9b3-4be0-989a-51c6330310de service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:38.661188 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f893e-a9b3-4be0-989a-51c6330310de service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:38.661502 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f893e-a9b3-4be0-989a-51c6330310de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:38.670431 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f893e-a9b3-4be0-989a-51c6330310de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:38.671935 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f893e-a9b3-4be0-989a-51c6330310de service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:38.680958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1411/2817] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:00:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:00:40.379152 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd3153c-0256-4289-80fb-83b0aa0eaf9e None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:40.380583 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-4dd3153c-0256-4289-80fb-83b0aa0eaf9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:40.437878 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:40.456083 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd3153c-0256-4289-80fb-83b0aa0eaf9e None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:40.460718 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1770e6f-e6e7-48b2-82e7-bf670e60c6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:40.461386 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1770e6f-e6e7-48b2-82e7-bf670e60c6d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:40.461665 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1770e6f-e6e7-48b2-82e7-bf670e60c6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:40.461936 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1770e6f-e6e7-48b2-82e7-bf670e60c6d3 admin admin] PATH_INFO: `/v3/users/8d6b5d8f849e42138b0925d12a470c21` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:40.463757 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1770e6f-e6e7-48b2-82e7-bf670e60c6d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d6b5d8f849e42138b0925d12a470c21)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:40.465793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1770e6f-e6e7-48b2-82e7-bf670e60c6d3 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:40.511434 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-a1770e6f-e6e7-48b2-82e7-bf670e60c6d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d6b5d8f849e42138b0925d12a470c21'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:40.518196 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-a1770e6f-e6e7-48b2-82e7-bf670e60c6d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d6b5d8f849e42138b0925d12a470c21'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:40.530283 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-a1770e6f-e6e7-48b2-82e7-bf670e60c6d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d6b5d8f849e42138b0925d12a470c21'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:40.530955 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-a1770e6f-e6e7-48b2-82e7-bf670e60c6d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d6b5d8f849e42138b0925d12a470c21'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:40.531563 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-a1770e6f-e6e7-48b2-82e7-bf670e60c6d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d6b5d8f849e42138b0925d12a470c21'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:40.544449 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1407/2818] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 23:00:40 2022] DELETE /identity/v3/users/8d6b5d8f849e42138b0925d12a470c21 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:40.549644 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc099b-6c98-4528-b68c-2b49a290a0c3 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:40.550387 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc099b-6c98-4528-b68c-2b49a290a0c3 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:40.550570 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc099b-6c98-4528-b68c-2b49a290a0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:40.570328 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-98bc099b-6c98-4528-b68c-2b49a290a0c3 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:40.571534 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-98bc099b-6c98-4528-b68c-2b49a290a0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:40.630692 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1412/2819] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:40.641379 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1770e6f-e6e7-48b2-82e7-bf670e60c6d3 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:40.642788 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-a1770e6f-e6e7-48b2-82e7-bf670e60c6d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:40.699505 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1770e6f-e6e7-48b2-82e7-bf670e60c6d3 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:40.703218 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80dcd5-ef37-4521-901d-6106fb0f06d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:40.703832 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80dcd5-ef37-4521-901d-6106fb0f06d1 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:40.704111 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80dcd5-ef37-4521-901d-6106fb0f06d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:40.704385 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80dcd5-ef37-4521-901d-6106fb0f06d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:40.713853 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de80dcd5-ef37-4521-901d-6106fb0f06d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:40.716196 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de80dcd5-ef37-4521-901d-6106fb0f06d1 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:40.730816 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1408/2820] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:00:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:00:41.009882 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc099b-6c98-4528-b68c-2b49a290a0c3 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:41.011268 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-98bc099b-6c98-4528-b68c-2b49a290a0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:41.026274 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc099b-6c98-4528-b68c-2b49a290a0c3 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:41.031066 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2334f-579f-4863-b3a8-923880ee4f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:41.031758 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2334f-579f-4863-b3a8-923880ee4f0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:41.032040 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2334f-579f-4863-b3a8-923880ee4f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:41.032316 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2334f-579f-4863-b3a8-923880ee4f0d admin admin] PATH_INFO: `/v3/projects/75b7f3845c004c02ac9964c459eb11b7` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:41.033486 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac2334f-579f-4863-b3a8-923880ee4f0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=75b7f3845c004c02ac9964c459eb11b7)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:41.035499 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac2334f-579f-4863-b3a8-923880ee4f0d admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:41.089577 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-2ac2334f-579f-4863-b3a8-923880ee4f0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75b7f3845c004c02ac9964c459eb11b7', 'request_id': 'req-2ac2334f-579f-4863-b3a8-923880ee4f0d'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:41.095061 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-2ac2334f-579f-4863-b3a8-923880ee4f0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75b7f3845c004c02ac9964c459eb11b7', 'request_id': 'req-2ac2334f-579f-4863-b3a8-923880ee4f0d'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:41.102427 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-2ac2334f-579f-4863-b3a8-923880ee4f0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75b7f3845c004c02ac9964c459eb11b7', 'request_id': 'req-2ac2334f-579f-4863-b3a8-923880ee4f0d'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:41.114396 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-2ac2334f-579f-4863-b3a8-923880ee4f0d admin admin] The token cache is being invalidate because project 75b7f3845c004c02ac9964c459eb11b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:00:41.114685 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-2ac2334f-579f-4863-b3a8-923880ee4f0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:41.115318 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-2ac2334f-579f-4863-b3a8-923880ee4f0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:41.117066 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1413/2821] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 23:00:41 2022] DELETE /identity/v3/projects/75b7f3845c004c02ac9964c459eb11b7 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:41.121693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80dcd5-ef37-4521-901d-6106fb0f06d1 service neutron] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:41.131168 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80dcd5-ef37-4521-901d-6106fb0f06d1 service neutron] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:41.191384 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719a673-b433-4604-bd0f-ad30248e8784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:41.192005 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719a673-b433-4604-bd0f-ad30248e8784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:41.192278 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719a673-b433-4604-bd0f-ad30248e8784 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:41.192568 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719a673-b433-4604-bd0f-ad30248e8784 admin admin] PATH_INFO: `/v3/users/c5325aeed76e40c098cd2b3d19781fa2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:41.193702 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1719a673-b433-4604-bd0f-ad30248e8784 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5325aeed76e40c098cd2b3d19781fa2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:41.195608 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1719a673-b433-4604-bd0f-ad30248e8784 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:41.208274 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:41.241457 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-1719a673-b433-4604-bd0f-ad30248e8784 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5325aeed76e40c098cd2b3d19781fa2'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:41.248890 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-1719a673-b433-4604-bd0f-ad30248e8784 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5325aeed76e40c098cd2b3d19781fa2'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:41.261705 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-1719a673-b433-4604-bd0f-ad30248e8784 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5325aeed76e40c098cd2b3d19781fa2'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:41.262396 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-1719a673-b433-4604-bd0f-ad30248e8784 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5325aeed76e40c098cd2b3d19781fa2'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:41.262996 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-1719a673-b433-4604-bd0f-ad30248e8784 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5325aeed76e40c098cd2b3d19781fa2'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:41.280213 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1409/2822] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 23:00:41 2022] DELETE /identity/v3/users/c5325aeed76e40c098cd2b3d19781fa2 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:41.567449 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2334f-579f-4863-b3a8-923880ee4f0d admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:41.568953 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-2ac2334f-579f-4863-b3a8-923880ee4f0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:41.583358 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2334f-579f-4863-b3a8-923880ee4f0d admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:41.641624 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cca142-6fc5-4704-85ea-1ee726614ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:41.642261 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cca142-6fc5-4704-85ea-1ee726614ad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:41.642546 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cca142-6fc5-4704-85ea-1ee726614ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:41.642823 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cca142-6fc5-4704-85ea-1ee726614ad3 admin admin] PATH_INFO: `/v3/projects/5eff7dc5b2a9416299cc33fe17258fd8` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:41.643964 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cca142-6fc5-4704-85ea-1ee726614ad3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eff7dc5b2a9416299cc33fe17258fd8)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:41.645927 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cca142-6fc5-4704-85ea-1ee726614ad3 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:41.647417 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:41.702217 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-41cca142-6fc5-4704-85ea-1ee726614ad3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eff7dc5b2a9416299cc33fe17258fd8', 'request_id': 'req-41cca142-6fc5-4704-85ea-1ee726614ad3'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:41.707851 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-41cca142-6fc5-4704-85ea-1ee726614ad3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eff7dc5b2a9416299cc33fe17258fd8', 'request_id': 'req-41cca142-6fc5-4704-85ea-1ee726614ad3'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:41.716702 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-41cca142-6fc5-4704-85ea-1ee726614ad3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eff7dc5b2a9416299cc33fe17258fd8', 'request_id': 'req-41cca142-6fc5-4704-85ea-1ee726614ad3'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:41.729383 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-41cca142-6fc5-4704-85ea-1ee726614ad3 admin admin] The token cache is being invalidate because project 5eff7dc5b2a9416299cc33fe17258fd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:00:41.729675 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-41cca142-6fc5-4704-85ea-1ee726614ad3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:41.730340 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-41cca142-6fc5-4704-85ea-1ee726614ad3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:41.731841 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1414/2823] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 23:00:41 2022] DELETE /identity/v3/projects/5eff7dc5b2a9416299cc33fe17258fd8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:41.736032 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719a673-b433-4604-bd0f-ad30248e8784 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:41.737426 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-1719a673-b433-4604-bd0f-ad30248e8784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:41.750916 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719a673-b433-4604-bd0f-ad30248e8784 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:41.809088 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2663243-5ddc-4306-b7e4-eca25002f3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:41.809751 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2663243-5ddc-4306-b7e4-eca25002f3b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:41.810030 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2663243-5ddc-4306-b7e4-eca25002f3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:41.810303 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2663243-5ddc-4306-b7e4-eca25002f3b9 admin admin] PATH_INFO: `/v3/users/6e03e4f8e2f34867bed146ed69ce94ab` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:41.811420 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2663243-5ddc-4306-b7e4-eca25002f3b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e03e4f8e2f34867bed146ed69ce94ab)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:41.813360 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2663243-5ddc-4306-b7e4-eca25002f3b9 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:41.860306 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f2663243-5ddc-4306-b7e4-eca25002f3b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e03e4f8e2f34867bed146ed69ce94ab'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:41.867488 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f2663243-5ddc-4306-b7e4-eca25002f3b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e03e4f8e2f34867bed146ed69ce94ab'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:41.879447 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f2663243-5ddc-4306-b7e4-eca25002f3b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e03e4f8e2f34867bed146ed69ce94ab'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:41.880120 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f2663243-5ddc-4306-b7e4-eca25002f3b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e03e4f8e2f34867bed146ed69ce94ab'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:41.880718 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-f2663243-5ddc-4306-b7e4-eca25002f3b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e03e4f8e2f34867bed146ed69ce94ab'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:41.894391 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1410/2824] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 23:00:41 2022] DELETE /identity/v3/users/6e03e4f8e2f34867bed146ed69ce94ab => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:42.177146 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cca142-6fc5-4704-85ea-1ee726614ad3 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:42.179079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-41cca142-6fc5-4704-85ea-1ee726614ad3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:42.192506 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cca142-6fc5-4704-85ea-1ee726614ad3 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:42.254195 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea1161-1ea8-45fc-b848-a06f880b2251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:42.254841 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea1161-1ea8-45fc-b848-a06f880b2251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:42.255127 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea1161-1ea8-45fc-b848-a06f880b2251 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:42.255410 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea1161-1ea8-45fc-b848-a06f880b2251 admin admin] PATH_INFO: `/v3/projects/a8c99bbf674e4578b985fc7e68cd08c8` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:42.256554 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ea1161-1ea8-45fc-b848-a06f880b2251 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8c99bbf674e4578b985fc7e68cd08c8)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:42.258511 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ea1161-1ea8-45fc-b848-a06f880b2251 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:42.308005 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-22ea1161-1ea8-45fc-b848-a06f880b2251 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8c99bbf674e4578b985fc7e68cd08c8', 'request_id': 'req-22ea1161-1ea8-45fc-b848-a06f880b2251'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:42.313732 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-22ea1161-1ea8-45fc-b848-a06f880b2251 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8c99bbf674e4578b985fc7e68cd08c8', 'request_id': 'req-22ea1161-1ea8-45fc-b848-a06f880b2251'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:42.323251 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-22ea1161-1ea8-45fc-b848-a06f880b2251 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8c99bbf674e4578b985fc7e68cd08c8', 'request_id': 'req-22ea1161-1ea8-45fc-b848-a06f880b2251'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:42.335162 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-22ea1161-1ea8-45fc-b848-a06f880b2251 admin admin] The token cache is being invalidate because project a8c99bbf674e4578b985fc7e68cd08c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:00:42.335450 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-22ea1161-1ea8-45fc-b848-a06f880b2251 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:42.336079 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-22ea1161-1ea8-45fc-b848-a06f880b2251 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:00:42.338240 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1415/2825] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 23:00:42 2022] DELETE /identity/v3/projects/a8c99bbf674e4578b985fc7e68cd08c8 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:42.343337 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831717b4-b310-45ba-88f6-80ec4de965eb None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:42.343628 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831717b4-b310-45ba-88f6-80ec4de965eb None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:42.343903 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831717b4-b310-45ba-88f6-80ec4de965eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:42.364368 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-831717b4-b310-45ba-88f6-80ec4de965eb None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:42.365969 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-831717b4-b310-45ba-88f6-80ec4de965eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:42.371185 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:42.429805 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1411/2826] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:00:42.439850 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cf5ef-3094-4330-a053-81b219969e27 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:42.440138 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cf5ef-3094-4330-a053-81b219969e27 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:42.440411 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cf5ef-3094-4330-a053-81b219969e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:42.460042 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-c68cf5ef-3094-4330-a053-81b219969e27 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:42.461251 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-c68cf5ef-3094-4330-a053-81b219969e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:42.469455 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1416/2827] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:42.473646 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831717b4-b310-45ba-88f6-80ec4de965eb None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:42.484221 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831717b4-b310-45ba-88f6-80ec4de965eb None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:42.488748 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada27ad-d9af-4f51-989e-b7a0b2c0cdff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:42.489373 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada27ad-d9af-4f51-989e-b7a0b2c0cdff admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:42.489650 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada27ad-d9af-4f51-989e-b7a0b2c0cdff admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:42.489916 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada27ad-d9af-4f51-989e-b7a0b2c0cdff admin admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:42.490549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada27ad-d9af-4f51-989e-b7a0b2c0cdff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:00:42.490846 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada27ad-d9af-4f51-989e-b7a0b2c0cdff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:42.492693 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada27ad-d9af-4f51-989e-b7a0b2c0cdff admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:42.504467 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1412/2828] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 23:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:00:42.508632 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68cf5ef-3094-4330-a053-81b219969e27 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:42.518565 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68cf5ef-3094-4330-a053-81b219969e27 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:42.523342 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a6093-fbc0-4402-bb23-bee189fc29eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:42.523976 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a6093-fbc0-4402-bb23-bee189fc29eb admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:42.524256 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a6093-fbc0-4402-bb23-bee189fc29eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:42.524627 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a6093-fbc0-4402-bb23-bee189fc29eb admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:42.525374 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a6093-fbc0-4402-bb23-bee189fc29eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:42.527271 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a6093-fbc0-4402-bb23-bee189fc29eb admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:42.538677 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:42.560601 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1417/2829] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 23:00:42 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:00:42.561824 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada27ad-d9af-4f51-989e-b7a0b2c0cdff admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:42.572263 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada27ad-d9af-4f51-989e-b7a0b2c0cdff admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:42.631390 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32fb5f-5771-4aaa-9ac9-31945aaf9202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:42.631994 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32fb5f-5771-4aaa-9ac9-31945aaf9202 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:42.632276 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32fb5f-5771-4aaa-9ac9-31945aaf9202 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:42.632541 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32fb5f-5771-4aaa-9ac9-31945aaf9202 admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:42.633159 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f32fb5f-5771-4aaa-9ac9-31945aaf9202 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:42.635765 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f32fb5f-5771-4aaa-9ac9-31945aaf9202 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:42.656517 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1413/2830] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 23:00:42 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:00:42.660577 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a6093-fbc0-4402-bb23-bee189fc29eb admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:42.671583 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a6093-fbc0-4402-bb23-bee189fc29eb admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:42.676246 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a233484-2901-4902-847b-c4d02be2fce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:42.676839 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a233484-2901-4902-847b-c4d02be2fce7 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:42.677120 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a233484-2901-4902-847b-c4d02be2fce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:42.677438 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a233484-2901-4902-847b-c4d02be2fce7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:42.678106 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a233484-2901-4902-847b-c4d02be2fce7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:42.679698 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a233484-2901-4902-847b-c4d02be2fce7 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:42.690008 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1418/2831] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:00:42.693996 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32fb5f-5771-4aaa-9ac9-31945aaf9202 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:42.704561 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32fb5f-5771-4aaa-9ac9-31945aaf9202 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:42.709232 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a97f2b-4457-42f9-a6ca-1b8f37d4db60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:42.709866 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a97f2b-4457-42f9-a6ca-1b8f37d4db60 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:42.710148 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a97f2b-4457-42f9-a6ca-1b8f37d4db60 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:42.710422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a97f2b-4457-42f9-a6ca-1b8f37d4db60 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:42.711055 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a97f2b-4457-42f9-a6ca-1b8f37d4db60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:42.713529 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a97f2b-4457-42f9-a6ca-1b8f37d4db60 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:42.724012 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1414/2832] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:00:42.728616 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a233484-2901-4902-847b-c4d02be2fce7 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:42.738304 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a233484-2901-4902-847b-c4d02be2fce7 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:42.743258 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e4ec89-4717-4c33-9770-4f0eff09a811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:42.743915 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4ec89-4717-4c33-9770-4f0eff09a811 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:42.744221 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4ec89-4717-4c33-9770-4f0eff09a811 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:42.744499 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4ec89-4717-4c33-9770-4f0eff09a811 admin admin] PATH_INFO: `/v3/projects/19b7bdf29e5d4fad87f8991179fc46ea/users/bc550ece01ba4d36adc64dbb963d38d4/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:42.759746 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e4ec89-4717-4c33-9770-4f0eff09a811 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19b7bdf29e5d4fad87f8991179fc46ea, user_id=bc550ece01ba4d36adc64dbb963d38d4, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:42.762396 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e4ec89-4717-4c33-9770-4f0eff09a811 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:42.776611 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1419/2833] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 23:00:42 2022] PUT /identity/v3/projects/19b7bdf29e5d4fad87f8991179fc46ea/users/bc550ece01ba4d36adc64dbb963d38d4/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:42.781696 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da65c1-bf17-4af2-ace2-511ba696115b None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:42.782155 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da65c1-bf17-4af2-ace2-511ba696115b None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:42.782253 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da65c1-bf17-4af2-ace2-511ba696115b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:42.801944 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-24da65c1-bf17-4af2-ace2-511ba696115b None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:42.803175 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-24da65c1-bf17-4af2-ace2-511ba696115b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:42.863561 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1415/2834] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:42.873600 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e4ec89-4717-4c33-9770-4f0eff09a811 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:42.875324 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-96e4ec89-4717-4c33-9770-4f0eff09a811 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:42.933287 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e4ec89-4717-4c33-9770-4f0eff09a811 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:42.936914 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e2cff-d28b-43be-8712-7b343573ca2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:42.937541 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e2cff-d28b-43be-8712-7b343573ca2b service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:42.937823 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e2cff-d28b-43be-8712-7b343573ca2b service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:42.938097 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e2cff-d28b-43be-8712-7b343573ca2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:42.947046 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94e2cff-d28b-43be-8712-7b343573ca2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:42.948476 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94e2cff-d28b-43be-8712-7b343573ca2b service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:42.957843 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1420/2835] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:00:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:00:47.118047 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168390e0-ec99-42f7-b73e-687361b62e16 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:47.118597 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168390e0-ec99-42f7-b73e-687361b62e16 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:47.118597 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168390e0-ec99-42f7-b73e-687361b62e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:47.150261 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-168390e0-ec99-42f7-b73e-687361b62e16 None None] MFA Rules not processed for user `bc550ece01ba4d36adc64dbb963d38d4`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:47.151803 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-168390e0-ec99-42f7-b73e-687361b62e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:47.156221 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:47.227030 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1416/2836] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:47.232531 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e2cff-d28b-43be-8712-7b343573ca2b service neutron] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:47.243564 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e2cff-d28b-43be-8712-7b343573ca2b service neutron] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:47.248402 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a0039-21da-4300-8357-ca283c3ef04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:47.249050 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a0039-21da-4300-8357-ca283c3ef04b admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:47.249389 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a0039-21da-4300-8357-ca283c3ef04b admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:47.249759 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a0039-21da-4300-8357-ca283c3ef04b admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:47.250303 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a0039-21da-4300-8357-ca283c3ef04b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:47.252228 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a0039-21da-4300-8357-ca283c3ef04b admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:47.286237 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1421/2837] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 23:00:47 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:00:47.291143 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168390e0-ec99-42f7-b73e-687361b62e16 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:47.302251 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168390e0-ec99-42f7-b73e-687361b62e16 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:47.362319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ff2b9-4030-46e6-9c8e-f80c532c29d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:47.362939 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ff2b9-4030-46e6-9c8e-f80c532c29d8 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:47.363224 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ff2b9-4030-46e6-9c8e-f80c532c29d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:47.363501 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ff2b9-4030-46e6-9c8e-f80c532c29d8 admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:47.364135 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ff2b9-4030-46e6-9c8e-f80c532c29d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:47.366368 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ff2b9-4030-46e6-9c8e-f80c532c29d8 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:47.394381 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1417/2838] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 23:00:47 2022] POST /identity/v3/users => generated 295 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:00:47.398514 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a0039-21da-4300-8357-ca283c3ef04b admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:47.411442 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a0039-21da-4300-8357-ca283c3ef04b admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:47.417651 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ff2b9-4030-46e6-9c8e-f80c532c29d8 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:47.419315 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-fc6ff2b9-4030-46e6-9c8e-f80c532c29d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:47.427750 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82b280-0ff0-4585-805f-4e3dd0c9a389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:47.428627 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82b280-0ff0-4585-805f-4e3dd0c9a389 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:47.432356 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82b280-0ff0-4585-805f-4e3dd0c9a389 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:47.432791 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82b280-0ff0-4585-805f-4e3dd0c9a389 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:47.433974 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d82b280-0ff0-4585-805f-4e3dd0c9a389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:47.437925 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d82b280-0ff0-4585-805f-4e3dd0c9a389 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:47.457419 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1422/2839] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:00:47.458879 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82b280-0ff0-4585-805f-4e3dd0c9a389 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:47.471613 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82b280-0ff0-4585-805f-4e3dd0c9a389 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:47.476763 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b5380-e77d-4a84-a1f6-09a892d07488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:47.478156 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b5380-e77d-4a84-a1f6-09a892d07488 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:47.478156 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b5380-e77d-4a84-a1f6-09a892d07488 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:47.478399 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b5380-e77d-4a84-a1f6-09a892d07488 admin admin] PATH_INFO: `/v3/projects/fd7c35a50310465c9b216d0346f268ff/users/6df3ba0eeb6b48dd9e6680b7a5c4cfd2/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:47.495351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232b5380-e77d-4a84-a1f6-09a892d07488 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd7c35a50310465c9b216d0346f268ff, user_id=6df3ba0eeb6b48dd9e6680b7a5c4cfd2, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:47.498075 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232b5380-e77d-4a84-a1f6-09a892d07488 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:47.509125 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ff2b9-4030-46e6-9c8e-f80c532c29d8 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:47.515379 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1423/2840] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 23:00:47 2022] PUT /identity/v3/projects/fd7c35a50310465c9b216d0346f268ff/users/6df3ba0eeb6b48dd9e6680b7a5c4cfd2/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:47.519047 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b5380-e77d-4a84-a1f6-09a892d07488 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:47.553646 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:47.588344 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b5380-e77d-4a84-a1f6-09a892d07488 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:47.590471 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783cb6b4-4325-471c-b54c-c504fa0be7a5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f46dee0f73e49eab54c11556e3b86bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:47.591089 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783cb6b4-4325-471c-b54c-c504fa0be7a5 service placement] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:47.591414 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783cb6b4-4325-471c-b54c-c504fa0be7a5 service placement] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:47.591684 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783cb6b4-4325-471c-b54c-c504fa0be7a5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:47.593466 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b96b55-3d8e-4713-bf39-318c372cf1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:47.593656 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-783cb6b4-4325-471c-b54c-c504fa0be7a5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:47.594179 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b96b55-3d8e-4713-bf39-318c372cf1b2 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:47.594473 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b96b55-3d8e-4713-bf39-318c372cf1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:47.594741 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b96b55-3d8e-4713-bf39-318c372cf1b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:47.595382 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b96b55-3d8e-4713-bf39-318c372cf1b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:47.596985 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b96b55-3d8e-4713-bf39-318c372cf1b2 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:47.599677 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:00:47.609004 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783cb6b4-4325-471c-b54c-c504fa0be7a5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:47.610836 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783cb6b4-4325-471c-b54c-c504fa0be7a5 service placement] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:47.619780 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1424/2841] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 23:00:47.620314 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b96b55-3d8e-4713-bf39-318c372cf1b2 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:47.632170 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b96b55-3d8e-4713-bf39-318c372cf1b2 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:47.638308 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087750e4-0daa-40e0-81de-bafa01bb2b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:47.639215 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087750e4-0daa-40e0-81de-bafa01bb2b6c admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:47.639499 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087750e4-0daa-40e0-81de-bafa01bb2b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:47.639760 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1418/2842] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:00:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:00:47.639891 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087750e4-0daa-40e0-81de-bafa01bb2b6c admin admin] PATH_INFO: `/v3/projects/fd7c35a50310465c9b216d0346f268ff/users/6df3ba0eeb6b48dd9e6680b7a5c4cfd2/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:47.641855 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087750e4-0daa-40e0-81de-bafa01bb2b6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd7c35a50310465c9b216d0346f268ff, user_id=6df3ba0eeb6b48dd9e6680b7a5c4cfd2, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:47.644528 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087750e4-0daa-40e0-81de-bafa01bb2b6c admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:47.657165 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1425/2843] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 23:00:47 2022] PUT /identity/v3/projects/fd7c35a50310465c9b216d0346f268ff/users/6df3ba0eeb6b48dd9e6680b7a5c4cfd2/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:00:52.478638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83488f9-4fe1-46e1-992e-fef99c288032 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:52.479295 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83488f9-4fe1-46e1-992e-fef99c288032 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:52.479295 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83488f9-4fe1-46e1-992e-fef99c288032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:52.514086 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-e83488f9-4fe1-46e1-992e-fef99c288032 None None] MFA Rules not processed for user `6df3ba0eeb6b48dd9e6680b7a5c4cfd2`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:00:52.515374 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-e83488f9-4fe1-46e1-992e-fef99c288032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:52.623883 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1419/2844] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:00:52 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:00:52.638308 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087750e4-0daa-40e0-81de-bafa01bb2b6c admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:00:52.640862 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-087750e4-0daa-40e0-81de-bafa01bb2b6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:00:52.728545 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087750e4-0daa-40e0-81de-bafa01bb2b6c admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:00:52.734692 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b33dd-27b9-4ec7-b35a-751000e29991 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:00:52.735567 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b33dd-27b9-4ec7-b35a-751000e29991 service nova] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:00:52.736052 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b33dd-27b9-4ec7-b35a-751000e29991 service nova] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:00:52.736323 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b33dd-27b9-4ec7-b35a-751000e29991 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:00:52.846309 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735b33dd-27b9-4ec7-b35a-751000e29991 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:00:52.850231 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735b33dd-27b9-4ec7-b35a-751000e29991 service nova] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:00:52.864750 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1426/2845] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:00:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:01:02.093788 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83488f9-4fe1-46e1-992e-fef99c288032 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:02.095220 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-e83488f9-4fe1-46e1-992e-fef99c288032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:01:02.164309 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83488f9-4fe1-46e1-992e-fef99c288032 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:02.168947 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f13584-e66e-4a13-8b42-78f38378ca98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:02.169612 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f13584-e66e-4a13-8b42-78f38378ca98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:02.169889 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f13584-e66e-4a13-8b42-78f38378ca98 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:02.170160 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f13584-e66e-4a13-8b42-78f38378ca98 admin admin] PATH_INFO: `/v3/users/36bed85f60024b539767dc72a3f68eab` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:02.171292 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f13584-e66e-4a13-8b42-78f38378ca98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36bed85f60024b539767dc72a3f68eab)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:02.173596 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f13584-e66e-4a13-8b42-78f38378ca98 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:02.227762 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-93f13584-e66e-4a13-8b42-78f38378ca98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36bed85f60024b539767dc72a3f68eab'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:02.233230 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:01:02.235698 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-93f13584-e66e-4a13-8b42-78f38378ca98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36bed85f60024b539767dc72a3f68eab'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:02.249105 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-93f13584-e66e-4a13-8b42-78f38378ca98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36bed85f60024b539767dc72a3f68eab'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:02.250590 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-93f13584-e66e-4a13-8b42-78f38378ca98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36bed85f60024b539767dc72a3f68eab'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:02.251374 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-93f13584-e66e-4a13-8b42-78f38378ca98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36bed85f60024b539767dc72a3f68eab'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:02.264696 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1420/2846] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 23:01:02 2022] DELETE /identity/v3/users/36bed85f60024b539767dc72a3f68eab => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:01:02.270430 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bcaa1c-ccb1-4877-8acb-24dfae28299f None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:02.270957 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bcaa1c-ccb1-4877-8acb-24dfae28299f None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:02.271089 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bcaa1c-ccb1-4877-8acb-24dfae28299f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:02.290221 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-96bcaa1c-ccb1-4877-8acb-24dfae28299f None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:01:02.291436 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-96bcaa1c-ccb1-4877-8acb-24dfae28299f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:01:02.353375 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1427/2847] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:01:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:01:02.363763 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f13584-e66e-4a13-8b42-78f38378ca98 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:02.365830 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-93f13584-e66e-4a13-8b42-78f38378ca98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:01:02.425118 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f13584-e66e-4a13-8b42-78f38378ca98 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:02.429084 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f180fe1f-6a19-482f-9f9c-ed8958a5c5f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:02.429721 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180fe1f-6a19-482f-9f9c-ed8958a5c5f2 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:02.429998 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180fe1f-6a19-482f-9f9c-ed8958a5c5f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:02.430275 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180fe1f-6a19-482f-9f9c-ed8958a5c5f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:02.441914 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f180fe1f-6a19-482f-9f9c-ed8958a5c5f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:02.443582 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f180fe1f-6a19-482f-9f9c-ed8958a5c5f2 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:02.454495 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1421/2848] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:01:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:01:02.742055 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bcaa1c-ccb1-4877-8acb-24dfae28299f None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:02.743570 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-96bcaa1c-ccb1-4877-8acb-24dfae28299f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:01:02.757675 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bcaa1c-ccb1-4877-8acb-24dfae28299f None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:02.762677 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc157f-3258-409e-ba63-857d768ec583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:02.763328 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc157f-3258-409e-ba63-857d768ec583 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:02.763612 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc157f-3258-409e-ba63-857d768ec583 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:02.763966 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc157f-3258-409e-ba63-857d768ec583 admin admin] PATH_INFO: `/v3/projects/4306706a5b3e49bdad9c75b899b470d0` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:02.765205 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cc157f-3258-409e-ba63-857d768ec583 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4306706a5b3e49bdad9c75b899b470d0)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:02.767390 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cc157f-3258-409e-ba63-857d768ec583 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:02.792183 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:01:02.820530 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-06cc157f-3258-409e-ba63-857d768ec583 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4306706a5b3e49bdad9c75b899b470d0', 'request_id': 'req-06cc157f-3258-409e-ba63-857d768ec583'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:02.826039 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-06cc157f-3258-409e-ba63-857d768ec583 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4306706a5b3e49bdad9c75b899b470d0', 'request_id': 'req-06cc157f-3258-409e-ba63-857d768ec583'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:02.835170 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-06cc157f-3258-409e-ba63-857d768ec583 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4306706a5b3e49bdad9c75b899b470d0', 'request_id': 'req-06cc157f-3258-409e-ba63-857d768ec583'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:02.846563 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-06cc157f-3258-409e-ba63-857d768ec583 admin admin] The token cache is being invalidate because project 4306706a5b3e49bdad9c75b899b470d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:01:02.846850 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-06cc157f-3258-409e-ba63-857d768ec583 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:02.849582 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-06cc157f-3258-409e-ba63-857d768ec583 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:02.851292 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1428/2849] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 23:01:02 2022] DELETE /identity/v3/projects/4306706a5b3e49bdad9c75b899b470d0 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:01:02.856339 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8635e33-4527-4ed7-83e7-370c973c4524 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:02.856823 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8635e33-4527-4ed7-83e7-370c973c4524 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:02.857106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8635e33-4527-4ed7-83e7-370c973c4524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:02.880368 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-a8635e33-4527-4ed7-83e7-370c973c4524 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:01:02.881621 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-a8635e33-4527-4ed7-83e7-370c973c4524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:01:02.944617 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1422/2850] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:01:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:01:02.952536 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd32a7d-b654-40ec-af8c-975af78a22a9 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:02.953042 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd32a7d-b654-40ec-af8c-975af78a22a9 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:02.953599 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd32a7d-b654-40ec-af8c-975af78a22a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:02.974983 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-3dd32a7d-b654-40ec-af8c-975af78a22a9 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:01:02.976467 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-3dd32a7d-b654-40ec-af8c-975af78a22a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:01:02.985344 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1429/2851] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:01:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:01:02.989944 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8635e33-4527-4ed7-83e7-370c973c4524 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:03.000847 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8635e33-4527-4ed7-83e7-370c973c4524 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:03.005528 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829723c-c30b-4391-9101-7fb43f405b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:03.006133 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829723c-c30b-4391-9101-7fb43f405b87 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:03.006407 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829723c-c30b-4391-9101-7fb43f405b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:03.006677 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829723c-c30b-4391-9101-7fb43f405b87 admin admin] PATH_INFO: `/v3/domains` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:03.007313 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b829723c-c30b-4391-9101-7fb43f405b87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93439) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:01:03.007611 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b829723c-c30b-4391-9101-7fb43f405b87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:03.009190 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b829723c-c30b-4391-9101-7fb43f405b87 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:03.021559 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1423/2852] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 23:01:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:01:03.026723 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd32a7d-b654-40ec-af8c-975af78a22a9 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:03.037403 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd32a7d-b654-40ec-af8c-975af78a22a9 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:03.042167 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0b78e-4f8a-4db7-8bc8-d488a105820f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:03.042787 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0b78e-4f8a-4db7-8bc8-d488a105820f admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:03.043063 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0b78e-4f8a-4db7-8bc8-d488a105820f admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:03.043329 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0b78e-4f8a-4db7-8bc8-d488a105820f admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:03.043948 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0b78e-4f8a-4db7-8bc8-d488a105820f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:03.045879 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0b78e-4f8a-4db7-8bc8-d488a105820f admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:03.076064 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1430/2853] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 23:01:03 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:01:03.080548 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829723c-c30b-4391-9101-7fb43f405b87 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:03.090514 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829723c-c30b-4391-9101-7fb43f405b87 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:03.152312 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31139230-1e05-49eb-82b0-d0b84dcca438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:03.152988 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31139230-1e05-49eb-82b0-d0b84dcca438 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:03.153198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31139230-1e05-49eb-82b0-d0b84dcca438 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:03.153500 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31139230-1e05-49eb-82b0-d0b84dcca438 admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:03.154129 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31139230-1e05-49eb-82b0-d0b84dcca438 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:03.156957 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31139230-1e05-49eb-82b0-d0b84dcca438 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:03.166119 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:01:03.182479 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1424/2854] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 23:01:03 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:01:03.186520 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0b78e-4f8a-4db7-8bc8-d488a105820f admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:03.197186 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0b78e-4f8a-4db7-8bc8-d488a105820f admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:03.201864 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e6a01-f31d-4ece-ba24-2dd2ead48d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:03.202515 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e6a01-f31d-4ece-ba24-2dd2ead48d9a admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:03.202802 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e6a01-f31d-4ece-ba24-2dd2ead48d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:03.203080 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e6a01-f31d-4ece-ba24-2dd2ead48d9a admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:03.203715 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4e6a01-f31d-4ece-ba24-2dd2ead48d9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:03.205342 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4e6a01-f31d-4ece-ba24-2dd2ead48d9a admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:03.215705 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1431/2855] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:01:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:01:03.220410 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31139230-1e05-49eb-82b0-d0b84dcca438 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:03.230671 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31139230-1e05-49eb-82b0-d0b84dcca438 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:03.235936 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b4df8-ac17-4ead-82e2-b80e2340f5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:03.236534 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b4df8-ac17-4ead-82e2-b80e2340f5e0 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:03.236811 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b4df8-ac17-4ead-82e2-b80e2340f5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:03.237082 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b4df8-ac17-4ead-82e2-b80e2340f5e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:03.237752 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b4df8-ac17-4ead-82e2-b80e2340f5e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:03.239368 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b4df8-ac17-4ead-82e2-b80e2340f5e0 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:03.250354 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1425/2856] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:01:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:01:03.254544 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e6a01-f31d-4ece-ba24-2dd2ead48d9a admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:03.264713 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e6a01-f31d-4ece-ba24-2dd2ead48d9a admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:03.270325 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318b16b-b013-4c7b-a973-6b6cde695cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:03.271003 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318b16b-b013-4c7b-a973-6b6cde695cf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:03.271571 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318b16b-b013-4c7b-a973-6b6cde695cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:03.271571 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318b16b-b013-4c7b-a973-6b6cde695cf0 admin admin] PATH_INFO: `/v3/projects/1e80b29bb2ff408c97768af7b59f89a5/users/fb8d639d576f43d3a3a39876776f49d4/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:03.287197 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8318b16b-b013-4c7b-a973-6b6cde695cf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e80b29bb2ff408c97768af7b59f89a5, user_id=fb8d639d576f43d3a3a39876776f49d4, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:03.289934 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8318b16b-b013-4c7b-a973-6b6cde695cf0 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:03.303987 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1432/2857] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 23:01:03 2022] PUT /identity/v3/projects/1e80b29bb2ff408c97768af7b59f89a5/users/fb8d639d576f43d3a3a39876776f49d4/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:01:03.315102 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7a584-d1c6-4bf4-9d99-5490556f5611 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:03.315401 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7a584-d1c6-4bf4-9d99-5490556f5611 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:03.315701 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7a584-d1c6-4bf4-9d99-5490556f5611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:03.346564 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-b5b7a584-d1c6-4bf4-9d99-5490556f5611 None None] MFA Rules not processed for user `fb8d639d576f43d3a3a39876776f49d4`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:01:03.347897 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-b5b7a584-d1c6-4bf4-9d99-5490556f5611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:01:03.415363 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1426/2858] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:01:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:01:03.419861 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318b16b-b013-4c7b-a973-6b6cde695cf0 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:03.486780 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318b16b-b013-4c7b-a973-6b6cde695cf0 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:03.492017 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee0c05-438f-40f9-9dc5-32654f7440c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:03.492627 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee0c05-438f-40f9-9dc5-32654f7440c2 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:03.492907 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee0c05-438f-40f9-9dc5-32654f7440c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:03.493206 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee0c05-438f-40f9-9dc5-32654f7440c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:03.493850 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ee0c05-438f-40f9-9dc5-32654f7440c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:03.495811 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ee0c05-438f-40f9-9dc5-32654f7440c2 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:03.526093 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1433/2859] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 23:01:03 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:01:03.527331 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b7a584-d1c6-4bf4-9d99-5490556f5611 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:03.538954 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b7a584-d1c6-4bf4-9d99-5490556f5611 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:03.601430 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfedb04-1888-4d07-b1b4-60dc553a3b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:03.602054 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfedb04-1888-4d07-b1b4-60dc553a3b94 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:03.602335 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfedb04-1888-4d07-b1b4-60dc553a3b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:03.602608 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfedb04-1888-4d07-b1b4-60dc553a3b94 admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:03.603246 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfedb04-1888-4d07-b1b4-60dc553a3b94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:03.605186 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfedb04-1888-4d07-b1b4-60dc553a3b94 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:03.627939 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1427/2860] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 23:01:03 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:01:03.632006 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee0c05-438f-40f9-9dc5-32654f7440c2 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:03.643675 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee0c05-438f-40f9-9dc5-32654f7440c2 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:03.648716 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c973e-e76e-4b6b-8367-1ecdbaa36029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:03.649374 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c973e-e76e-4b6b-8367-1ecdbaa36029 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:03.649657 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c973e-e76e-4b6b-8367-1ecdbaa36029 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:03.649954 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c973e-e76e-4b6b-8367-1ecdbaa36029 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:03.650597 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c973e-e76e-4b6b-8367-1ecdbaa36029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:03.652225 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c973e-e76e-4b6b-8367-1ecdbaa36029 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:03.664636 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1434/2861] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:01:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:01:03.668712 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfedb04-1888-4d07-b1b4-60dc553a3b94 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:03.680137 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfedb04-1888-4d07-b1b4-60dc553a3b94 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:03.685019 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69bbb83-a62d-4cda-9e3a-b6944e9e171a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:03.685582 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69bbb83-a62d-4cda-9e3a-b6944e9e171a admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:03.685855 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69bbb83-a62d-4cda-9e3a-b6944e9e171a admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:03.686125 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69bbb83-a62d-4cda-9e3a-b6944e9e171a admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:03.686791 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69bbb83-a62d-4cda-9e3a-b6944e9e171a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:03.688405 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69bbb83-a62d-4cda-9e3a-b6944e9e171a admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:03.699476 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1428/2862] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:01:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:01:03.703812 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c973e-e76e-4b6b-8367-1ecdbaa36029 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:03.714669 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c973e-e76e-4b6b-8367-1ecdbaa36029 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:03.721075 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67662a97-2467-425f-b650-9efb251f5419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:03.721785 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67662a97-2467-425f-b650-9efb251f5419 admin admin] REQUEST_METHOD: `PUT` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:03.722072 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67662a97-2467-425f-b650-9efb251f5419 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:03.722560 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67662a97-2467-425f-b650-9efb251f5419 admin admin] PATH_INFO: `/v3/projects/c3d889447cab4ae583e78a280a633b7e/users/e169a69205904ae3942f187da87e4af3/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:03.736269 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:01:03.741745 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67662a97-2467-425f-b650-9efb251f5419 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3d889447cab4ae583e78a280a633b7e, user_id=e169a69205904ae3942f187da87e4af3, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:03.748773 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67662a97-2467-425f-b650-9efb251f5419 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:03.764013 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1435/2863] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 23:01:03 2022] PUT /identity/v3/projects/c3d889447cab4ae583e78a280a633b7e/users/e169a69205904ae3942f187da87e4af3/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:01:03.776312 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b34636-e993-423d-8ed5-757d5b1feae1 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:03.776604 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b34636-e993-423d-8ed5-757d5b1feae1 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:03.776885 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b34636-e993-423d-8ed5-757d5b1feae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:03.806776 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-12b34636-e993-423d-8ed5-757d5b1feae1 None None] MFA Rules not processed for user `e169a69205904ae3942f187da87e4af3`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:01:03.808057 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-12b34636-e993-423d-8ed5-757d5b1feae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:01:03.846841 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:01:03.877230 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1429/2864] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:01:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:01:03.881609 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67662a97-2467-425f-b650-9efb251f5419 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:03.947298 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67662a97-2467-425f-b650-9efb251f5419 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:03.952018 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325a053-15ab-4ce4-bfbb-a5e815a5c888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:03.952779 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325a053-15ab-4ce4-bfbb-a5e815a5c888 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:03.953108 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325a053-15ab-4ce4-bfbb-a5e815a5c888 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:03.953353 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325a053-15ab-4ce4-bfbb-a5e815a5c888 admin admin] PATH_INFO: `/v3/projects` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:03.953988 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a325a053-15ab-4ce4-bfbb-a5e815a5c888 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:03.955901 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a325a053-15ab-4ce4-bfbb-a5e815a5c888 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:03.983359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1436/2865] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 23:01:03 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 23:01:03.988196 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b34636-e993-423d-8ed5-757d5b1feae1 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:03.999262 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b34636-e993-423d-8ed5-757d5b1feae1 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:04.059652 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf98db-3f09-42ce-aa10-3f003f9dd547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:04.060261 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf98db-3f09-42ce-aa10-3f003f9dd547 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:04.060541 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf98db-3f09-42ce-aa10-3f003f9dd547 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:04.060812 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf98db-3f09-42ce-aa10-3f003f9dd547 admin admin] PATH_INFO: `/v3/users` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:04.061478 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbf98db-3f09-42ce-aa10-3f003f9dd547 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:04.063719 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbf98db-3f09-42ce-aa10-3f003f9dd547 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:04.086582 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1430/2866] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 23:01:03 2022] POST /identity/v3/users => generated 298 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:01:04.090442 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325a053-15ab-4ce4-bfbb-a5e815a5c888 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:04.101548 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325a053-15ab-4ce4-bfbb-a5e815a5c888 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:04.106290 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22432054-1e58-4b36-94df-b7a50eaa68fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:04.106899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22432054-1e58-4b36-94df-b7a50eaa68fd admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:04.107179 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22432054-1e58-4b36-94df-b7a50eaa68fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:04.107456 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22432054-1e58-4b36-94df-b7a50eaa68fd admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:04.108093 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22432054-1e58-4b36-94df-b7a50eaa68fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:04.110500 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22432054-1e58-4b36-94df-b7a50eaa68fd admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:04.122642 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1437/2867] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:01:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:01:04.128378 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf98db-3f09-42ce-aa10-3f003f9dd547 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:04.138561 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf98db-3f09-42ce-aa10-3f003f9dd547 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:04.143475 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071583dd-7bdf-429d-8397-c732f2d84726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:04.144124 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071583dd-7bdf-429d-8397-c732f2d84726 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:04.144404 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071583dd-7bdf-429d-8397-c732f2d84726 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:04.144709 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071583dd-7bdf-429d-8397-c732f2d84726 admin admin] PATH_INFO: `/v3/projects/2f0d5dc683c84b9087a635655572f19b/users/80f9c2357bcd41608b1987691bcbc5f4/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:04.160638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071583dd-7bdf-429d-8397-c732f2d84726 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f0d5dc683c84b9087a635655572f19b, user_id=80f9c2357bcd41608b1987691bcbc5f4, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:04.163359 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071583dd-7bdf-429d-8397-c732f2d84726 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:04.178406 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1431/2868] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 23:01:04 2022] PUT /identity/v3/projects/2f0d5dc683c84b9087a635655572f19b/users/80f9c2357bcd41608b1987691bcbc5f4/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:01:04.182696 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22432054-1e58-4b36-94df-b7a50eaa68fd admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:04.193627 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22432054-1e58-4b36-94df-b7a50eaa68fd admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:04.256800 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63583c51-e020-4e1a-a3c4-910e28fb9f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:04.257625 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63583c51-e020-4e1a-a3c4-910e28fb9f58 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:04.258018 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63583c51-e020-4e1a-a3c4-910e28fb9f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:04.258229 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63583c51-e020-4e1a-a3c4-910e28fb9f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:04.258908 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63583c51-e020-4e1a-a3c4-910e28fb9f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:04.260558 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63583c51-e020-4e1a-a3c4-910e28fb9f58 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:04.271203 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1438/2869] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:01:04 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:01:04.275198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071583dd-7bdf-429d-8397-c732f2d84726 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:04.287145 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071583dd-7bdf-429d-8397-c732f2d84726 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:04.291971 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c050e3-9275-43c4-afbd-0ffe5aecd48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:04.292650 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c050e3-9275-43c4-afbd-0ffe5aecd48d admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:04.292925 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c050e3-9275-43c4-afbd-0ffe5aecd48d admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:04.293196 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c050e3-9275-43c4-afbd-0ffe5aecd48d admin admin] PATH_INFO: `/v3/projects/2f0d5dc683c84b9087a635655572f19b/users/80f9c2357bcd41608b1987691bcbc5f4/roles/6641e95fdc1f4c098ebc372bb692d227` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:04.295143 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c050e3-9275-43c4-afbd-0ffe5aecd48d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f0d5dc683c84b9087a635655572f19b, user_id=80f9c2357bcd41608b1987691bcbc5f4, role_id=6641e95fdc1f4c098ebc372bb692d227)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:04.298586 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c050e3-9275-43c4-afbd-0ffe5aecd48d admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:04.309552 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1432/2870] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 23:01:04 2022] PUT /identity/v3/projects/2f0d5dc683c84b9087a635655572f19b/users/80f9c2357bcd41608b1987691bcbc5f4/roles/6641e95fdc1f4c098ebc372bb692d227 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:01:04.320707 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf3242-fbe2-4e98-aa94-8735adcc9556 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:04.321014 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf3242-fbe2-4e98-aa94-8735adcc9556 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:04.321334 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf3242-fbe2-4e98-aa94-8735adcc9556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:04.351438 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-d7cf3242-fbe2-4e98-aa94-8735adcc9556 None None] MFA Rules not processed for user `80f9c2357bcd41608b1987691bcbc5f4`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:01:04.352665 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-d7cf3242-fbe2-4e98-aa94-8735adcc9556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:01:04.404711 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:01:04.430872 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1439/2871] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:01:04 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:01:04.442601 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c050e3-9275-43c4-afbd-0ffe5aecd48d admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:04.444351 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-22c050e3-9275-43c4-afbd-0ffe5aecd48d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:01:04.504915 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c050e3-9275-43c4-afbd-0ffe5aecd48d admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:04.508668 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bacba9-1702-4114-8fcd-b079bfb5ebf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:04.509299 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bacba9-1702-4114-8fcd-b079bfb5ebf8 service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:04.509591 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bacba9-1702-4114-8fcd-b079bfb5ebf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:04.509866 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bacba9-1702-4114-8fcd-b079bfb5ebf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:04.561365 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:01:04.575885 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bacba9-1702-4114-8fcd-b079bfb5ebf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:04.578273 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bacba9-1702-4114-8fcd-b079bfb5ebf8 service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:04.587986 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1433/2872] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:01:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:01:06.114541 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf3242-fbe2-4e98-aa94-8735adcc9556 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:06.115999 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-d7cf3242-fbe2-4e98-aa94-8735adcc9556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:01:06.128470 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf3242-fbe2-4e98-aa94-8735adcc9556 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:06.132112 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b9553-87de-4b55-96a2-dd93761dc30e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:06.132715 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b9553-87de-4b55-96a2-dd93761dc30e service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:06.132999 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b9553-87de-4b55-96a2-dd93761dc30e service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:06.133289 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b9553-87de-4b55-96a2-dd93761dc30e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:06.192958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07b9553-87de-4b55-96a2-dd93761dc30e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:06.194356 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07b9553-87de-4b55-96a2-dd93761dc30e service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:06.204313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1440/2873] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:01:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:01:06.927064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bacba9-1702-4114-8fcd-b079bfb5ebf8 service cinder] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:06.936006 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bacba9-1702-4114-8fcd-b079bfb5ebf8 service cinder] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:06.939925 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef7734-f270-4e90-a230-78177c04d80d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:06.940523 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef7734-f270-4e90-a230-78177c04d80d service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:06.940803 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef7734-f270-4e90-a230-78177c04d80d service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:06.941075 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef7734-f270-4e90-a230-78177c04d80d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:06.950658 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbef7734-f270-4e90-a230-78177c04d80d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:06.952117 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbef7734-f270-4e90-a230-78177c04d80d service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:06.963162 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1434/2874] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:01:06 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:01:10.928680 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b9553-87de-4b55-96a2-dd93761dc30e service cinder] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:10.991530 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b9553-87de-4b55-96a2-dd93761dc30e service cinder] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:10.996088 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15e1d1-7686-4aa7-b994-70036c6a6c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:10.996710 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15e1d1-7686-4aa7-b994-70036c6a6c62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:10.996987 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15e1d1-7686-4aa7-b994-70036c6a6c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:10.997258 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15e1d1-7686-4aa7-b994-70036c6a6c62 admin admin] PATH_INFO: `/v3/users/fb8d639d576f43d3a3a39876776f49d4` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:10.998389 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb15e1d1-7686-4aa7-b994-70036c6a6c62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb8d639d576f43d3a3a39876776f49d4)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:11.000336 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb15e1d1-7686-4aa7-b994-70036c6a6c62 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:11.047174 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-bb15e1d1-7686-4aa7-b994-70036c6a6c62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb8d639d576f43d3a3a39876776f49d4'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:11.054375 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-bb15e1d1-7686-4aa7-b994-70036c6a6c62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb8d639d576f43d3a3a39876776f49d4'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:11.061706 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-bb15e1d1-7686-4aa7-b994-70036c6a6c62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb8d639d576f43d3a3a39876776f49d4'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:11.062730 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-bb15e1d1-7686-4aa7-b994-70036c6a6c62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb8d639d576f43d3a3a39876776f49d4'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:11.063427 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-bb15e1d1-7686-4aa7-b994-70036c6a6c62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb8d639d576f43d3a3a39876776f49d4'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:11.080908 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1441/2875] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 23:01:10 2022] DELETE /identity/v3/users/fb8d639d576f43d3a3a39876776f49d4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:01:11.085375 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88219701-b7e7-4136-9970-0e7479cb6250 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:11.085873 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88219701-b7e7-4136-9970-0e7479cb6250 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:11.085981 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88219701-b7e7-4136-9970-0e7479cb6250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:11.104670 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-88219701-b7e7-4136-9970-0e7479cb6250 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:01:11.105915 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-88219701-b7e7-4136-9970-0e7479cb6250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:01:11.164038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1435/2876] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:01:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:01:11.174220 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15e1d1-7686-4aa7-b994-70036c6a6c62 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:11.175862 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-bb15e1d1-7686-4aa7-b994-70036c6a6c62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:01:11.183183 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:01:11.236090 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15e1d1-7686-4aa7-b994-70036c6a6c62 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:11.240077 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7836c25a-718f-45bd-bdec-820d215fd92f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:11.240692 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836c25a-718f-45bd-bdec-820d215fd92f service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:11.241034 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836c25a-718f-45bd-bdec-820d215fd92f service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:11.241331 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836c25a-718f-45bd-bdec-820d215fd92f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:11.253177 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7836c25a-718f-45bd-bdec-820d215fd92f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:11.254679 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7836c25a-718f-45bd-bdec-820d215fd92f service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:11.264825 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1442/2877] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:01:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:01:11.545243 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88219701-b7e7-4136-9970-0e7479cb6250 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:11.547208 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-88219701-b7e7-4136-9970-0e7479cb6250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:01:11.560675 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88219701-b7e7-4136-9970-0e7479cb6250 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:11.566541 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce8a45-2038-4a93-b3f2-9183885a20ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:11.567192 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce8a45-2038-4a93-b3f2-9183885a20ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:11.567537 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce8a45-2038-4a93-b3f2-9183885a20ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:11.567744 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce8a45-2038-4a93-b3f2-9183885a20ab admin admin] PATH_INFO: `/v3/projects/1e80b29bb2ff408c97768af7b59f89a5` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:11.569875 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce8a45-2038-4a93-b3f2-9183885a20ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e80b29bb2ff408c97768af7b59f89a5)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:11.572506 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce8a45-2038-4a93-b3f2-9183885a20ab admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:11.623268 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-46ce8a45-2038-4a93-b3f2-9183885a20ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e80b29bb2ff408c97768af7b59f89a5', 'request_id': 'req-46ce8a45-2038-4a93-b3f2-9183885a20ab'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:11.631311 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-46ce8a45-2038-4a93-b3f2-9183885a20ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e80b29bb2ff408c97768af7b59f89a5', 'request_id': 'req-46ce8a45-2038-4a93-b3f2-9183885a20ab'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:11.636779 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-46ce8a45-2038-4a93-b3f2-9183885a20ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e80b29bb2ff408c97768af7b59f89a5', 'request_id': 'req-46ce8a45-2038-4a93-b3f2-9183885a20ab'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:11.647822 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-46ce8a45-2038-4a93-b3f2-9183885a20ab admin admin] The token cache is being invalidate because project 1e80b29bb2ff408c97768af7b59f89a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:01:11.648105 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-46ce8a45-2038-4a93-b3f2-9183885a20ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:11.648742 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-46ce8a45-2038-4a93-b3f2-9183885a20ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:11.650254 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1436/2878] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 23:01:11 2022] DELETE /identity/v3/projects/1e80b29bb2ff408c97768af7b59f89a5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:01:11.654399 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7836c25a-718f-45bd-bdec-820d215fd92f service neutron] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:11.664378 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7836c25a-718f-45bd-bdec-820d215fd92f service neutron] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:11.720075 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e5b34f-435a-4630-a44a-83cdcf5b6c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:11.720623 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5b34f-435a-4630-a44a-83cdcf5b6c10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:11.720903 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5b34f-435a-4630-a44a-83cdcf5b6c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:11.721175 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5b34f-435a-4630-a44a-83cdcf5b6c10 admin admin] PATH_INFO: `/v3/users/e169a69205904ae3942f187da87e4af3` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:11.722349 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e5b34f-435a-4630-a44a-83cdcf5b6c10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e169a69205904ae3942f187da87e4af3)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:11.724267 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e5b34f-435a-4630-a44a-83cdcf5b6c10 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:11.770420 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-50e5b34f-435a-4630-a44a-83cdcf5b6c10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e169a69205904ae3942f187da87e4af3'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:11.778055 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-50e5b34f-435a-4630-a44a-83cdcf5b6c10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e169a69205904ae3942f187da87e4af3'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:11.785141 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-50e5b34f-435a-4630-a44a-83cdcf5b6c10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e169a69205904ae3942f187da87e4af3'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:11.785815 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-50e5b34f-435a-4630-a44a-83cdcf5b6c10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e169a69205904ae3942f187da87e4af3'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:11.786415 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-50e5b34f-435a-4630-a44a-83cdcf5b6c10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e169a69205904ae3942f187da87e4af3'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:11.804519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1443/2879] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 23:01:11 2022] DELETE /identity/v3/users/e169a69205904ae3942f187da87e4af3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 23:01:12.098583 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce8a45-2038-4a93-b3f2-9183885a20ab admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:12.100101 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-46ce8a45-2038-4a93-b3f2-9183885a20ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:01:12.114055 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce8a45-2038-4a93-b3f2-9183885a20ab admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:12.164140 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:01:12.173042 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43475477-87d2-434d-997e-65c0570feb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:12.173786 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43475477-87d2-434d-997e-65c0570feb52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:12.174242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43475477-87d2-434d-997e-65c0570feb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:12.174721 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43475477-87d2-434d-997e-65c0570feb52 admin admin] PATH_INFO: `/v3/projects/c3d889447cab4ae583e78a280a633b7e` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:12.176227 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43475477-87d2-434d-997e-65c0570feb52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3d889447cab4ae583e78a280a633b7e)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:12.178517 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43475477-87d2-434d-997e-65c0570feb52 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:12.231913 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-43475477-87d2-434d-997e-65c0570feb52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3d889447cab4ae583e78a280a633b7e', 'request_id': 'req-43475477-87d2-434d-997e-65c0570feb52'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:12.239952 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-43475477-87d2-434d-997e-65c0570feb52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3d889447cab4ae583e78a280a633b7e', 'request_id': 'req-43475477-87d2-434d-997e-65c0570feb52'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:12.249427 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-43475477-87d2-434d-997e-65c0570feb52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3d889447cab4ae583e78a280a633b7e', 'request_id': 'req-43475477-87d2-434d-997e-65c0570feb52'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:12.261411 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-43475477-87d2-434d-997e-65c0570feb52 admin admin] The token cache is being invalidate because project c3d889447cab4ae583e78a280a633b7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:01:12.261703 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-43475477-87d2-434d-997e-65c0570feb52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:12.262389 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-43475477-87d2-434d-997e-65c0570feb52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:12.264008 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1437/2880] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 23:01:12 2022] DELETE /identity/v3/projects/c3d889447cab4ae583e78a280a633b7e => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:01:12.269491 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e5b34f-435a-4630-a44a-83cdcf5b6c10 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:12.271600 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-50e5b34f-435a-4630-a44a-83cdcf5b6c10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:01:12.285837 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e5b34f-435a-4630-a44a-83cdcf5b6c10 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:12.310106 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:01:12.347852 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6fcbbc-1ddf-4d22-a157-be87d29da724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:12.348631 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6fcbbc-1ddf-4d22-a157-be87d29da724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:12.348968 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6fcbbc-1ddf-4d22-a157-be87d29da724 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:12.349407 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6fcbbc-1ddf-4d22-a157-be87d29da724 admin admin] PATH_INFO: `/v3/users/80f9c2357bcd41608b1987691bcbc5f4` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:12.350829 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6fcbbc-1ddf-4d22-a157-be87d29da724 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80f9c2357bcd41608b1987691bcbc5f4)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:12.353201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6fcbbc-1ddf-4d22-a157-be87d29da724 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:12.411369 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-4a6fcbbc-1ddf-4d22-a157-be87d29da724 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80f9c2357bcd41608b1987691bcbc5f4'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:12.422693 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-4a6fcbbc-1ddf-4d22-a157-be87d29da724 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80f9c2357bcd41608b1987691bcbc5f4'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:12.430345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-4a6fcbbc-1ddf-4d22-a157-be87d29da724 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80f9c2357bcd41608b1987691bcbc5f4'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:12.431572 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-4a6fcbbc-1ddf-4d22-a157-be87d29da724 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80f9c2357bcd41608b1987691bcbc5f4'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:12.431572 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-4a6fcbbc-1ddf-4d22-a157-be87d29da724 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80f9c2357bcd41608b1987691bcbc5f4'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:12.448661 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1444/2881] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 23:01:12 2022] DELETE /identity/v3/users/80f9c2357bcd41608b1987691bcbc5f4 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:01:12.729996 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43475477-87d2-434d-997e-65c0570feb52 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:12.731457 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-43475477-87d2-434d-997e-65c0570feb52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:01:12.750091 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43475477-87d2-434d-997e-65c0570feb52 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:12.814068 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c5276e-98c5-4226-afa0-b1baa59e4f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:12.814641 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5276e-98c5-4226-afa0-b1baa59e4f75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:12.814935 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5276e-98c5-4226-afa0-b1baa59e4f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:12.815210 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5276e-98c5-4226-afa0-b1baa59e4f75 admin admin] PATH_INFO: `/v3/projects/2f0d5dc683c84b9087a635655572f19b` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:12.816424 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c5276e-98c5-4226-afa0-b1baa59e4f75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f0d5dc683c84b9087a635655572f19b)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:12.818439 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c5276e-98c5-4226-afa0-b1baa59e4f75 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:12.873795 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e4c5276e-98c5-4226-afa0-b1baa59e4f75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f0d5dc683c84b9087a635655572f19b', 'request_id': 'req-e4c5276e-98c5-4226-afa0-b1baa59e4f75'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:12.882984 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e4c5276e-98c5-4226-afa0-b1baa59e4f75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f0d5dc683c84b9087a635655572f19b', 'request_id': 'req-e4c5276e-98c5-4226-afa0-b1baa59e4f75'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:12.889296 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e4c5276e-98c5-4226-afa0-b1baa59e4f75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f0d5dc683c84b9087a635655572f19b', 'request_id': 'req-e4c5276e-98c5-4226-afa0-b1baa59e4f75'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:12.901598 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e4c5276e-98c5-4226-afa0-b1baa59e4f75 admin admin] The token cache is being invalidate because project 2f0d5dc683c84b9087a635655572f19b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93439) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:01:12.901891 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e4c5276e-98c5-4226-afa0-b1baa59e4f75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:12.902533 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-e4c5276e-98c5-4226-afa0-b1baa59e4f75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:12.904003 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1438/2882] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 23:01:12 2022] DELETE /identity/v3/projects/2f0d5dc683c84b9087a635655572f19b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:01:12.909090 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ab681-eb96-4834-9c18-0252fded3f1b None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:12.909413 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ab681-eb96-4834-9c18-0252fded3f1b None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:12.909692 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ab681-eb96-4834-9c18-0252fded3f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:12.930318 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-3b2ab681-eb96-4834-9c18-0252fded3f1b None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:01:12.931516 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-3b2ab681-eb96-4834-9c18-0252fded3f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:01:12.991207 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1445/2883] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:01:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:01:13.000176 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73328550-1eac-4292-8a58-f925d28862ed None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:13.000613 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73328550-1eac-4292-8a58-f925d28862ed None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:13.000892 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73328550-1eac-4292-8a58-f925d28862ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:13.021670 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-73328550-1eac-4292-8a58-f925d28862ed None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:01:13.022899 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-73328550-1eac-4292-8a58-f925d28862ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:01:13.031551 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1439/2884] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:01:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:01:13.035674 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ab681-eb96-4834-9c18-0252fded3f1b None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:13.046060 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ab681-eb96-4834-9c18-0252fded3f1b None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:13.050758 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68805984-50a3-4800-8ea0-c9d267358686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:13.051351 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68805984-50a3-4800-8ea0-c9d267358686 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:13.051627 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68805984-50a3-4800-8ea0-c9d267358686 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:13.051896 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68805984-50a3-4800-8ea0-c9d267358686 admin admin] PATH_INFO: `/v3/domains` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:13.052527 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68805984-50a3-4800-8ea0-c9d267358686 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93438) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 23:01:13.052823 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68805984-50a3-4800-8ea0-c9d267358686 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:13.054441 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68805984-50a3-4800-8ea0-c9d267358686 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:13.066237 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1446/2885] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 10 23:01:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 23:01:13.071033 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73328550-1eac-4292-8a58-f925d28862ed None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:13.083351 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73328550-1eac-4292-8a58-f925d28862ed None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:13.088083 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee36bb89-1d36-4959-85aa-7c415e67fc34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:13.088710 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36bb89-1d36-4959-85aa-7c415e67fc34 admin admin] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:13.088992 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36bb89-1d36-4959-85aa-7c415e67fc34 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:13.089264 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36bb89-1d36-4959-85aa-7c415e67fc34 admin admin] PATH_INFO: `/v3/projects` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:13.089907 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee36bb89-1d36-4959-85aa-7c415e67fc34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:13.091844 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee36bb89-1d36-4959-85aa-7c415e67fc34 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:13.098012 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:01:13.121489 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1440/2886] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 10 23:01:13 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:01:13.125899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68805984-50a3-4800-8ea0-c9d267358686 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:13.135842 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68805984-50a3-4800-8ea0-c9d267358686 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:13.170815 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:01:13.203277 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf7991-6df3-46db-b3cf-168144109c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:13.203882 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf7991-6df3-46db-b3cf-168144109c39 admin admin] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:13.204164 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf7991-6df3-46db-b3cf-168144109c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:13.204444 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf7991-6df3-46db-b3cf-168144109c39 admin admin] PATH_INFO: `/v3/users` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:13.205063 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf7991-6df3-46db-b3cf-168144109c39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:13.207019 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf7991-6df3-46db-b3cf-168144109c39 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:13.230015 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1447/2887] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 10 23:01:13 2022] POST /identity/v3/users => generated 283 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 23:01:13.231145 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee36bb89-1d36-4959-85aa-7c415e67fc34 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:13.242449 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee36bb89-1d36-4959-85aa-7c415e67fc34 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:13.246994 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac28d31a-d017-40bc-81ce-23652e630253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:13.247583 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28d31a-d017-40bc-81ce-23652e630253 admin admin] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:13.247858 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28d31a-d017-40bc-81ce-23652e630253 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:13.248126 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28d31a-d017-40bc-81ce-23652e630253 admin admin] PATH_INFO: `/v3/roles` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:13.248753 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac28d31a-d017-40bc-81ce-23652e630253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:13.251064 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac28d31a-d017-40bc-81ce-23652e630253 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:13.260767 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1441/2888] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:01:13.264701 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf7991-6df3-46db-b3cf-168144109c39 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:13.274703 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf7991-6df3-46db-b3cf-168144109c39 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:13.279366 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c4d5fe-7c97-4d67-b6b3-d2e9841d47f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:13.280033 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4d5fe-7c97-4d67-b6b3-d2e9841d47f3 admin admin] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:13.280313 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4d5fe-7c97-4d67-b6b3-d2e9841d47f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:13.280583 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4d5fe-7c97-4d67-b6b3-d2e9841d47f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:13.281390 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c4d5fe-7c97-4d67-b6b3-d2e9841d47f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:13.283046 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c4d5fe-7c97-4d67-b6b3-d2e9841d47f3 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:13.292859 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1448/2889] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 10 23:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 23:01:13.297860 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac28d31a-d017-40bc-81ce-23652e630253 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:13.308160 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac28d31a-d017-40bc-81ce-23652e630253 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:13.313422 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef8a23-7c8c-4d6f-9783-92f1050dea61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:13.314066 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef8a23-7c8c-4d6f-9783-92f1050dea61 admin admin] REQUEST_METHOD: `PUT` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:13.314359 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef8a23-7c8c-4d6f-9783-92f1050dea61 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:13.314659 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef8a23-7c8c-4d6f-9783-92f1050dea61 admin admin] PATH_INFO: `/v3/projects/578e0f011ba34513bf8b0131f01481c4/users/c975404b51904ed083a999f2da7fc59c/roles/1085473c6e534f87812b05c7014f77c2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:13.329158 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ef8a23-7c8c-4d6f-9783-92f1050dea61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=578e0f011ba34513bf8b0131f01481c4, user_id=c975404b51904ed083a999f2da7fc59c, role_id=1085473c6e534f87812b05c7014f77c2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:13.332020 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ef8a23-7c8c-4d6f-9783-92f1050dea61 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:13.346149 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1442/2890] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 10 23:01:13 2022] PUT /identity/v3/projects/578e0f011ba34513bf8b0131f01481c4/users/c975404b51904ed083a999f2da7fc59c/roles/1085473c6e534f87812b05c7014f77c2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:01:13.356379 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1552fbf-4f8c-40c4-baf9-26ec52d7294d None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:13.356670 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1552fbf-4f8c-40c4-baf9-26ec52d7294d None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:13.356951 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1552fbf-4f8c-40c4-baf9-26ec52d7294d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:13.385413 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-a1552fbf-4f8c-40c4-baf9-26ec52d7294d None None] MFA Rules not processed for user `c975404b51904ed083a999f2da7fc59c`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:01:13.386633 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-a1552fbf-4f8c-40c4-baf9-26ec52d7294d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:01:13.456794 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1449/2891] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:01:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:01:13.467896 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef8a23-7c8c-4d6f-9783-92f1050dea61 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:13.470184 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-38ef8a23-7c8c-4d6f-9783-92f1050dea61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:01:13.525714 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef8a23-7c8c-4d6f-9783-92f1050dea61 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:13.529327 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e2b18-0e78-435f-9c39-c2d48954f508 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:13.529930 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e2b18-0e78-435f-9c39-c2d48954f508 service cinder] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:13.530204 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e2b18-0e78-435f-9c39-c2d48954f508 service cinder] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:13.530471 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e2b18-0e78-435f-9c39-c2d48954f508 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:13.539543 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e2b18-0e78-435f-9c39-c2d48954f508 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:13.540995 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e2b18-0e78-435f-9c39-c2d48954f508 service cinder] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:13.550495 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1443/2892] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:01:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:01:20.878370 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71da97-1745-4008-afee-784536198fd5 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:20.878876 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71da97-1745-4008-afee-784536198fd5 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:20.879007 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71da97-1745-4008-afee-784536198fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:20.897765 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-ee71da97-1745-4008-afee-784536198fd5 None None] MFA Rules not processed for user `bc550ece01ba4d36adc64dbb963d38d4`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:01:20.898972 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-ee71da97-1745-4008-afee-784536198fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:01:20.952686 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1450/2893] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:01:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:01:23.545928 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=46,keystone:INSERT=2 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:01:27.677972 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e2b18-0e78-435f-9c39-c2d48954f508 service cinder] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:27.679932 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-0f6e2b18-0e78-435f-9c39-c2d48954f508 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:01:27.774496 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e2b18-0e78-435f-9c39-c2d48954f508 service cinder] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:27.779954 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace6849-09b2-4c03-8717-36024055c030 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:27.780561 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace6849-09b2-4c03-8717-36024055c030 service nova] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:27.780839 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace6849-09b2-4c03-8717-36024055c030 service nova] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:27.781115 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace6849-09b2-4c03-8717-36024055c030 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:27.783580 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-6ace6849-09b2-4c03-8717-36024055c030 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:01:27.796956 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace6849-09b2-4c03-8717-36024055c030 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:27.799266 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace6849-09b2-4c03-8717-36024055c030 service nova] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:27.810256 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1444/2894] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:01:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:01:30.945888 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=76,keystone:INSERT=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:01:37.806330 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=34 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:01:39.118654 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee71da97-1745-4008-afee-784536198fd5 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:39.132981 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee71da97-1745-4008-afee-784536198fd5 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:39.139147 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb50705-6da0-4f9c-82f0-e0c692b35b01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:39.139930 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb50705-6da0-4f9c-82f0-e0c692b35b01 service nova] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:39.140899 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb50705-6da0-4f9c-82f0-e0c692b35b01 service nova] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:39.141180 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb50705-6da0-4f9c-82f0-e0c692b35b01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:39.143023 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-3cb50705-6da0-4f9c-82f0-e0c692b35b01 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:01:39.156946 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb50705-6da0-4f9c-82f0-e0c692b35b01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:39.158636 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb50705-6da0-4f9c-82f0-e0c692b35b01 service nova] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:39.169556 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1451/2895] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:01:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:01:41.936120 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace6849-09b2-4c03-8717-36024055c030 service nova] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:41.937595 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-6ace6849-09b2-4c03-8717-36024055c030 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:01:41.997242 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace6849-09b2-4c03-8717-36024055c030 service nova] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:42.001423 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2807a-95d9-4477-8977-14cbdd2a8cd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:42.002106 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2807a-95d9-4477-8977-14cbdd2a8cd2 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:42.002388 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2807a-95d9-4477-8977-14cbdd2a8cd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:42.002767 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2807a-95d9-4477-8977-14cbdd2a8cd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:42.004842 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-23a2807a-95d9-4477-8977-14cbdd2a8cd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:01:42.075014 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a2807a-95d9-4477-8977-14cbdd2a8cd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:42.076501 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a2807a-95d9-4477-8977-14cbdd2a8cd2 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:42.087047 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1445/2896] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:01:41 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:01:44.013494 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb50705-6da0-4f9c-82f0-e0c692b35b01 service nova] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:44.014976 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-3cb50705-6da0-4f9c-82f0-e0c692b35b01 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:01:44.029697 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb50705-6da0-4f9c-82f0-e0c692b35b01 service nova] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:44.036404 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031fc50f-d7a2-41c5-9d85-36a107a4d90b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:44.037020 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031fc50f-d7a2-41c5-9d85-36a107a4d90b service nova] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:44.037332 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031fc50f-d7a2-41c5-9d85-36a107a4d90b service nova] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:44.037615 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031fc50f-d7a2-41c5-9d85-36a107a4d90b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:44.039424 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-031fc50f-d7a2-41c5-9d85-36a107a4d90b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:01:44.053157 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031fc50f-d7a2-41c5-9d85-36a107a4d90b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:44.054807 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031fc50f-d7a2-41c5-9d85-36a107a4d90b service nova] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:44.066200 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1452/2897] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:01:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:01:52.082793 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=50 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:01:54.061173 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=12 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:01:59.295856 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2807a-95d9-4477-8977-14cbdd2a8cd2 service neutron] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:59.359065 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2807a-95d9-4477-8977-14cbdd2a8cd2 service neutron] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:59.364326 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844537ba-76db-4580-b7b8-04a8ce2c59fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:59.364945 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844537ba-76db-4580-b7b8-04a8ce2c59fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:59.365220 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844537ba-76db-4580-b7b8-04a8ce2c59fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:59.365528 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844537ba-76db-4580-b7b8-04a8ce2c59fa admin admin] PATH_INFO: `/v3/users/c975404b51904ed083a999f2da7fc59c` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:59.366631 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844537ba-76db-4580-b7b8-04a8ce2c59fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=c975404b51904ed083a999f2da7fc59c)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:59.368539 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844537ba-76db-4580-b7b8-04a8ce2c59fa admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:59.415460 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-844537ba-76db-4580-b7b8-04a8ce2c59fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c975404b51904ed083a999f2da7fc59c'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:59.422433 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-844537ba-76db-4580-b7b8-04a8ce2c59fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c975404b51904ed083a999f2da7fc59c'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:59.435683 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-844537ba-76db-4580-b7b8-04a8ce2c59fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c975404b51904ed083a999f2da7fc59c'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:59.436411 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-844537ba-76db-4580-b7b8-04a8ce2c59fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c975404b51904ed083a999f2da7fc59c'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:59.437015 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-844537ba-76db-4580-b7b8-04a8ce2c59fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c975404b51904ed083a999f2da7fc59c'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:59.451108 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1446/2898] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 23:01:59 2022] DELETE /identity/v3/users/c975404b51904ed083a999f2da7fc59c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:01:59.457039 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924901aa-8818-462c-b7fb-49d925a801e7 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:59.457615 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924901aa-8818-462c-b7fb-49d925a801e7 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:59.457742 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924901aa-8818-462c-b7fb-49d925a801e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:59.481064 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-924901aa-8818-462c-b7fb-49d925a801e7 None None] MFA Rules not processed for user `d91ed101f8354003baf64fb2ad037357`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:01:59.482322 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-924901aa-8818-462c-b7fb-49d925a801e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:01:59.541923 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1453/2899] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:01:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:01:59.553135 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844537ba-76db-4580-b7b8-04a8ce2c59fa admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:59.554643 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-844537ba-76db-4580-b7b8-04a8ce2c59fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:01:59.612714 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844537ba-76db-4580-b7b8-04a8ce2c59fa admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:59.616377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0d02f-ec30-465d-b21d-ea14b1cbb6be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:59.616973 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0d02f-ec30-465d-b21d-ea14b1cbb6be service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:59.617252 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0d02f-ec30-465d-b21d-ea14b1cbb6be service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:59.617550 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0d02f-ec30-465d-b21d-ea14b1cbb6be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:59.627156 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d0d02f-ec30-465d-b21d-ea14b1cbb6be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:59.628579 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d0d02f-ec30-465d-b21d-ea14b1cbb6be service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:59.637961 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1447/2900] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:01:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:01:59.900352 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924901aa-8818-462c-b7fb-49d925a801e7 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:01:59.901840 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-924901aa-8818-462c-b7fb-49d925a801e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:01:59.917769 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924901aa-8818-462c-b7fb-49d925a801e7 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:01:59.923885 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edbe458-489c-42c9-bf0b-1ff39e2f2554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:01:59.924709 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbe458-489c-42c9-bf0b-1ff39e2f2554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:01:59.925056 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbe458-489c-42c9-bf0b-1ff39e2f2554 admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:01:59.925259 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbe458-489c-42c9-bf0b-1ff39e2f2554 admin admin] PATH_INFO: `/v3/projects/578e0f011ba34513bf8b0131f01481c4` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:01:59.926488 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edbe458-489c-42c9-bf0b-1ff39e2f2554 admin admin] RBAC: Authorizing `identity:delete_project(project_id=578e0f011ba34513bf8b0131f01481c4)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:01:59.928571 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edbe458-489c-42c9-bf0b-1ff39e2f2554 admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:01:59.975490 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-9edbe458-489c-42c9-bf0b-1ff39e2f2554 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '578e0f011ba34513bf8b0131f01481c4', 'request_id': 'req-9edbe458-489c-42c9-bf0b-1ff39e2f2554'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:59.981111 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-9edbe458-489c-42c9-bf0b-1ff39e2f2554 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '578e0f011ba34513bf8b0131f01481c4', 'request_id': 'req-9edbe458-489c-42c9-bf0b-1ff39e2f2554'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:01:59.989766 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-9edbe458-489c-42c9-bf0b-1ff39e2f2554 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '578e0f011ba34513bf8b0131f01481c4', 'request_id': 'req-9edbe458-489c-42c9-bf0b-1ff39e2f2554'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:02:00.000716 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-9edbe458-489c-42c9-bf0b-1ff39e2f2554 admin admin] The token cache is being invalidate because project 578e0f011ba34513bf8b0131f01481c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:02:00.001043 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-9edbe458-489c-42c9-bf0b-1ff39e2f2554 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:02:00.001696 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-9edbe458-489c-42c9-bf0b-1ff39e2f2554 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:02:00.003169 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1454/2901] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 23:01:59 2022] DELETE /identity/v3/projects/578e0f011ba34513bf8b0131f01481c4 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:02:09.633814 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:02:09.997797 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:03:42.808362 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0d02f-ec30-465d-b21d-ea14b1cbb6be service neutron] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:03:42.809821 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-b4d0d02f-ec30-465d-b21d-ea14b1cbb6be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:03:42.821592 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0d02f-ec30-465d-b21d-ea14b1cbb6be service neutron] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:03:42.870177 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e389d0a6-d73c-4b5e-8377-26ac2e491e8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:03:42.870776 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389d0a6-d73c-4b5e-8377-26ac2e491e8f service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:03:42.871051 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389d0a6-d73c-4b5e-8377-26ac2e491e8f service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:03:42.871316 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389d0a6-d73c-4b5e-8377-26ac2e491e8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:03:42.873139 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-e389d0a6-d73c-4b5e-8377-26ac2e491e8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:03:42.882849 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e389d0a6-d73c-4b5e-8377-26ac2e491e8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:03:42.884270 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e389d0a6-d73c-4b5e-8377-26ac2e491e8f service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:03:42.892306 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1448/2902] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:03:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:03:52.888638 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=26 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:04:12.012793 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edbe458-489c-42c9-bf0b-1ff39e2f2554 admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:04:12.014250 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-9edbe458-489c-42c9-bf0b-1ff39e2f2554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:04:12.026539 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edbe458-489c-42c9-bf0b-1ff39e2f2554 admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:04:12.029934 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d6af1e-525b-4b02-8386-2484835e55e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:04:12.030523 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6af1e-525b-4b02-8386-2484835e55e9 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:04:12.030799 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6af1e-525b-4b02-8386-2484835e55e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:04:12.031070 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6af1e-525b-4b02-8386-2484835e55e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:04:12.032705 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-f1d6af1e-525b-4b02-8386-2484835e55e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:04:12.103500 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d6af1e-525b-4b02-8386-2484835e55e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:04:12.104930 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d6af1e-525b-4b02-8386-2484835e55e9 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:04:12.116405 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1455/2903] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:04:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:04:13.302513 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e389d0a6-d73c-4b5e-8377-26ac2e491e8f service neutron] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:04:13.311331 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e389d0a6-d73c-4b5e-8377-26ac2e491e8f service neutron] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:04:13.315026 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe3352f-54a7-47f7-92a0-875af95adb3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:04:13.315913 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3352f-54a7-47f7-92a0-875af95adb3a service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:04:13.316439 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3352f-54a7-47f7-92a0-875af95adb3a service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:04:13.316821 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3352f-54a7-47f7-92a0-875af95adb3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:04:13.318915 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-bfe3352f-54a7-47f7-92a0-875af95adb3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:04:13.336511 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe3352f-54a7-47f7-92a0-875af95adb3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:04:13.338707 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe3352f-54a7-47f7-92a0-875af95adb3a service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:04:13.353831 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1449/2904] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:04:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:04:15.699863 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb797f6-172b-4412-aed2-4f7b7084c8c3 None None] REQUEST_METHOD: `POST` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:04:15.700429 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb797f6-172b-4412-aed2-4f7b7084c8c3 None None] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:04:15.700429 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb797f6-172b-4412-aed2-4f7b7084c8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:04:15.720326 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.auth.core [None req-dfb797f6-172b-4412-aed2-4f7b7084c8c3 None None] MFA Rules not processed for user `bc550ece01ba4d36adc64dbb963d38d4`. Rule list: `[]` (Enabled: `True`). {{(pid=93438) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:04:15.721949 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-dfb797f6-172b-4412-aed2-4f7b7084c8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:04:15.781306 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1456/2905] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:04:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 23:04:16.102874 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe3352f-54a7-47f7-92a0-875af95adb3a service neutron] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:04:16.112023 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe3352f-54a7-47f7-92a0-875af95adb3a service neutron] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:04:16.116022 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b72efe5-41c8-4d8e-ab12-3d9482bf40a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:04:16.116618 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72efe5-41c8-4d8e-ab12-3d9482bf40a7 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:04:16.116920 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72efe5-41c8-4d8e-ab12-3d9482bf40a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:04:16.117192 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72efe5-41c8-4d8e-ab12-3d9482bf40a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:04:16.119039 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-3b72efe5-41c8-4d8e-ab12-3d9482bf40a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:04:16.128579 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b72efe5-41c8-4d8e-ab12-3d9482bf40a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:04:16.133691 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b72efe5-41c8-4d8e-ab12-3d9482bf40a7 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:04:16.142344 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1450/2906] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:04:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:04:18.865614 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb797f6-172b-4412-aed2-4f7b7084c8c3 None None] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:04:18.867029 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-dfb797f6-172b-4412-aed2-4f7b7084c8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:04:18.935813 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb797f6-172b-4412-aed2-4f7b7084c8c3 None None] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:04:18.941039 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e95996-4500-4c03-9465-aa8c7a1f5be3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f46dee0f73e49eab54c11556e3b86bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:04:18.941694 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e95996-4500-4c03-9465-aa8c7a1f5be3 service placement] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:04:18.941977 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e95996-4500-4c03-9465-aa8c7a1f5be3 service placement] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:04:18.942254 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e95996-4500-4c03-9465-aa8c7a1f5be3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:04:18.944633 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-02e95996-4500-4c03-9465-aa8c7a1f5be3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:04:18.956569 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e95996-4500-4c03-9465-aa8c7a1f5be3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:04:18.958185 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e95996-4500-4c03-9465-aa8c7a1f5be3 service placement] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:04:18.969076 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1457/2907] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:04:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:04:26.138737 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=12 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:04:28.964614 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=91 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:05:48.134771 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b72efe5-41c8-4d8e-ab12-3d9482bf40a7 service neutron] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:05:48.136222 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-3b72efe5-41c8-4d8e-ab12-3d9482bf40a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:48.182171 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b72efe5-41c8-4d8e-ab12-3d9482bf40a7 service neutron] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:05:48.187674 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067dc0f-5518-44de-930f-829e5f5bee96 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f46dee0f73e49eab54c11556e3b86bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:05:48.188273 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067dc0f-5518-44de-930f-829e5f5bee96 service placement] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:05:48.188548 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067dc0f-5518-44de-930f-829e5f5bee96 service placement] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:05:48.188819 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067dc0f-5518-44de-930f-829e5f5bee96 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:05:48.190513 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-a067dc0f-5518-44de-930f-829e5f5bee96 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:05:48.201815 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067dc0f-5518-44de-930f-829e5f5bee96 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:05:48.204015 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067dc0f-5518-44de-930f-829e5f5bee96 service placement] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:05:48.214473 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1451/2908] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:05:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:05:58.209867 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=19 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:06:55.995684 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e95996-4500-4c03-9465-aa8c7a1f5be3 service placement] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:06:55.997201 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-02e95996-4500-4c03-9465-aa8c7a1f5be3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:56.100888 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e95996-4500-4c03-9465-aa8c7a1f5be3 service placement] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:06:56.104522 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c923bb01-1286-48ec-9814-c2587ef4db2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7f0dd6c9e2d4315966e90cd3b927b13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:06:56.105419 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923bb01-1286-48ec-9814-c2587ef4db2a service cinder] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:06:56.105419 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923bb01-1286-48ec-9814-c2587ef4db2a service cinder] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:06:56.105676 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923bb01-1286-48ec-9814-c2587ef4db2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:06:56.107539 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-c923bb01-1286-48ec-9814-c2587ef4db2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:06:56.132153 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c923bb01-1286-48ec-9814-c2587ef4db2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:06:56.133594 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c923bb01-1286-48ec-9814-c2587ef4db2a service cinder] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:06:56.143359 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1458/2909] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:06:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:07:06.139334 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=54 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:07.181038 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7694378-ab7d-40ab-bd58-322c3ea0be33 None None] REQUEST_METHOD: `POST` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:07.181627 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7694378-ab7d-40ab-bd58-322c3ea0be33 None None] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:07.181627 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7694378-ab7d-40ab-bd58-322c3ea0be33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:07.201250 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.auth.core [None req-d7694378-ab7d-40ab-bd58-322c3ea0be33 None None] MFA Rules not processed for user `bc550ece01ba4d36adc64dbb963d38d4`. Rule list: `[]` (Enabled: `True`). {{(pid=93439) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 23:07:07.202481 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-d7694378-ab7d-40ab-bd58-322c3ea0be33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:07.210125 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1452/2910] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 10 23:07:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 23:07:07.529574 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c923bb01-1286-48ec-9814-c2587ef4db2a service cinder] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:07:07.549539 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c923bb01-1286-48ec-9814-c2587ef4db2a service cinder] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:07:07.553069 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd54c7ab-864a-42e3-88a9-1b5972b44894 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:07:07.553697 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd54c7ab-864a-42e3-88a9-1b5972b44894 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:07.553971 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd54c7ab-864a-42e3-88a9-1b5972b44894 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:07.554242 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd54c7ab-864a-42e3-88a9-1b5972b44894 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:07.555958 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-fd54c7ab-864a-42e3-88a9-1b5972b44894 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:07.577873 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd54c7ab-864a-42e3-88a9-1b5972b44894 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:07:07.579291 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd54c7ab-864a-42e3-88a9-1b5972b44894 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:07:07.587657 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1459/2911] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:07:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:07:10.404101 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7694378-ab7d-40ab-bd58-322c3ea0be33 None None] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:07:10.405890 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-d7694378-ab7d-40ab-bd58-322c3ea0be33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:10.469492 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7694378-ab7d-40ab-bd58-322c3ea0be33 None None] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:07:10.475169 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ccef3-b1dd-4e22-ae98-be433b5fc8c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:07:10.475849 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ccef3-b1dd-4e22-ae98-be433b5fc8c3 service nova] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:10.476198 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ccef3-b1dd-4e22-ae98-be433b5fc8c3 service nova] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:10.476542 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ccef3-b1dd-4e22-ae98-be433b5fc8c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:10.478361 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-8a0ccef3-b1dd-4e22-ae98-be433b5fc8c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:10.491477 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0ccef3-b1dd-4e22-ae98-be433b5fc8c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:07:10.493165 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0ccef3-b1dd-4e22-ae98-be433b5fc8c3 service nova] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:07:10.504172 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1453/2912] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:07:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:07:17.583836 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=15 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:20.499615 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=24 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:20.842335 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd54c7ab-864a-42e3-88a9-1b5972b44894 service neutron] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:07:20.856592 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd54c7ab-864a-42e3-88a9-1b5972b44894 service neutron] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:07:20.862463 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab066ac-d23b-41dd-8b51-947a2f280538 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:07:20.863115 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab066ac-d23b-41dd-8b51-947a2f280538 service nova] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:20.863399 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab066ac-d23b-41dd-8b51-947a2f280538 service nova] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:20.863673 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab066ac-d23b-41dd-8b51-947a2f280538 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:20.865474 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-0ab066ac-d23b-41dd-8b51-947a2f280538 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:20.878578 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab066ac-d23b-41dd-8b51-947a2f280538 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:07:20.880177 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab066ac-d23b-41dd-8b51-947a2f280538 service nova] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:07:20.891199 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1460/2913] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:07:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:07:24.887997 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ccef3-b1dd-4e22-ae98-be433b5fc8c3 service nova] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:07:24.897444 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ccef3-b1dd-4e22-ae98-be433b5fc8c3 service nova] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:07:24.901001 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb580d1f-3ea0-4a16-bd56-1e118993db16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:07:24.901625 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb580d1f-3ea0-4a16-bd56-1e118993db16 service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:07:24.901907 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb580d1f-3ea0-4a16-bd56-1e118993db16 service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:07:24.902187 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb580d1f-3ea0-4a16-bd56-1e118993db16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:07:24.904102 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-cb580d1f-3ea0-4a16-bd56-1e118993db16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:07:24.967471 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb580d1f-3ea0-4a16-bd56-1e118993db16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:07:24.968837 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb580d1f-3ea0-4a16-bd56-1e118993db16 service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:07:24.978725 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1454/2914] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:07:24 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:07:30.887083 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=6 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:07:34.974377 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=30 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:08:47.828749 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab066ac-d23b-41dd-8b51-947a2f280538 service nova] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:08:47.838236 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab066ac-d23b-41dd-8b51-947a2f280538 service nova] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:08:47.842095 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291dd40-dce6-4b74-bd6e-e6b477588548 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:08:47.842692 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291dd40-dce6-4b74-bd6e-e6b477588548 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:08:47.842973 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291dd40-dce6-4b74-bd6e-e6b477588548 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:08:47.843245 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291dd40-dce6-4b74-bd6e-e6b477588548 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:08:47.850549 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1291dd40-dce6-4b74-bd6e-e6b477588548 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:08:47.851990 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1291dd40-dce6-4b74-bd6e-e6b477588548 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:08:47.860904 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1461/2915] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:08:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:08:57.857161 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=6 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:09:45.337216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb580d1f-3ea0-4a16-bd56-1e118993db16 service neutron] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:09:45.346174 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb580d1f-3ea0-4a16-bd56-1e118993db16 service neutron] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:09:45.349850 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2211ecd5-c1e7-42e5-b2de-a2d2926c8d0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:09:45.350452 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2211ecd5-c1e7-42e5-b2de-a2d2926c8d0f service neutron] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:09:45.351045 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2211ecd5-c1e7-42e5-b2de-a2d2926c8d0f service neutron] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:09:45.351045 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2211ecd5-c1e7-42e5-b2de-a2d2926c8d0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:09:45.362644 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2211ecd5-c1e7-42e5-b2de-a2d2926c8d0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:09:45.364100 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2211ecd5-c1e7-42e5-b2de-a2d2926c8d0f service neutron] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:09:45.376965 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1455/2916] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:09:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:09:46.567296 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291dd40-dce6-4b74-bd6e-e6b477588548 service neutron] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:09:46.576825 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291dd40-dce6-4b74-bd6e-e6b477588548 service neutron] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:09:46.580499 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ea46d-e4e8-432f-bb1a-b9cbbf7841f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:09:46.581113 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ea46d-e4e8-432f-bb1a-b9cbbf7841f1 service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:09:46.581405 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ea46d-e4e8-432f-bb1a-b9cbbf7841f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:09:46.581680 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ea46d-e4e8-432f-bb1a-b9cbbf7841f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:09:46.593187 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43ea46d-e4e8-432f-bb1a-b9cbbf7841f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:09:46.594719 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43ea46d-e4e8-432f-bb1a-b9cbbf7841f1 service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:09:46.607176 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1462/2917] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:09:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:09:47.665748 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2211ecd5-c1e7-42e5-b2de-a2d2926c8d0f service neutron] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:09:47.679356 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2211ecd5-c1e7-42e5-b2de-a2d2926c8d0f service neutron] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:09:47.686689 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4214b9-8562-4114-9a47-b83310f4ba65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b22838a8ad3949e59cdbaaa1801cd07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:09:47.687303 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4214b9-8562-4114-9a47-b83310f4ba65 service nova] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:09:47.687578 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4214b9-8562-4114-9a47-b83310f4ba65 service nova] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:09:47.687849 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4214b9-8562-4114-9a47-b83310f4ba65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:09:47.689645 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-9e4214b9-8562-4114-9a47-b83310f4ba65 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:09:47.704654 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4214b9-8562-4114-9a47-b83310f4ba65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:09:47.706637 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4214b9-8562-4114-9a47-b83310f4ba65 service nova] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:09:47.718356 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1456/2918] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:09:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:09:56.467871 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ea46d-e4e8-432f-bb1a-b9cbbf7841f1 service neutron] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:09:56.476691 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ea46d-e4e8-432f-bb1a-b9cbbf7841f1 service neutron] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:09:56.480966 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32bdfe-442e-41fc-9572-b82a6c510a4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fcb5a674ff94b3d8f79efbbe4959069', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:09:56.481598 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32bdfe-442e-41fc-9572-b82a6c510a4c service neutron] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:09:56.481879 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32bdfe-442e-41fc-9572-b82a6c510a4c service neutron] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:09:56.482150 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32bdfe-442e-41fc-9572-b82a6c510a4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:09:56.483877 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-9f32bdfe-442e-41fc-9572-b82a6c510a4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:09:56.570921 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f32bdfe-442e-41fc-9572-b82a6c510a4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:09:56.573010 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f32bdfe-442e-41fc-9572-b82a6c510a4c service neutron] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:09:56.592412 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1463/2919] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:09:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 23:09:57.713549 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=12 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:09:59.763963 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4214b9-8562-4114-9a47-b83310f4ba65 service nova] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:09:59.776248 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4214b9-8562-4114-9a47-b83310f4ba65 service nova] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:09:59.781665 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ab289d-6462-4b4f-a878-0eeab3c1f7d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f46dee0f73e49eab54c11556e3b86bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:09:59.782396 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab289d-6462-4b4f-a878-0eeab3c1f7d1 service placement] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:09:59.782673 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab289d-6462-4b4f-a878-0eeab3c1f7d1 service placement] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:09:59.783140 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab289d-6462-4b4f-a878-0eeab3c1f7d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:09:59.784958 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-b0ab289d-6462-4b4f-a878-0eeab3c1f7d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:09:59.798646 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ab289d-6462-4b4f-a878-0eeab3c1f7d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:09:59.800314 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ab289d-6462-4b4f-a878-0eeab3c1f7d1 service placement] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:09:59.814351 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1457/2920] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:09:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:10:02.733478 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32bdfe-442e-41fc-9572-b82a6c510a4c service neutron] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:10:02.745919 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32bdfe-442e-41fc-9572-b82a6c510a4c service neutron] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:10:02.751169 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466ab3a9-539b-435c-9811-f8ba9ff0bc34 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f46dee0f73e49eab54c11556e3b86bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:10:02.751790 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ab3a9-539b-435c-9811-f8ba9ff0bc34 service placement] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:10:02.752072 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ab3a9-539b-435c-9811-f8ba9ff0bc34 service placement] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:10:02.752345 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ab3a9-539b-435c-9811-f8ba9ff0bc34 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:10:02.754164 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-466ab3a9-539b-435c-9811-f8ba9ff0bc34 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:10:02.767573 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466ab3a9-539b-435c-9811-f8ba9ff0bc34 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:10:02.769196 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466ab3a9-539b-435c-9811-f8ba9ff0bc34 service placement] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:10:02.781768 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1464/2921] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:10:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:10:02.806164 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ab289d-6462-4b4f-a878-0eeab3c1f7d1 service placement] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:10:02.813040 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-b0ab289d-6462-4b4f-a878-0eeab3c1f7d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:10:02.834109 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ab289d-6462-4b4f-a878-0eeab3c1f7d1 service placement] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:10:02.839043 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f0dc2c-6928-45d5-9023-dadee7bf7966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:10:02.839669 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0dc2c-6928-45d5-9023-dadee7bf7966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:10:02.839952 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0dc2c-6928-45d5-9023-dadee7bf7966 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:10:02.840219 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0dc2c-6928-45d5-9023-dadee7bf7966 admin admin] PATH_INFO: `/v3/users/bc550ece01ba4d36adc64dbb963d38d4` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:10:02.841358 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f0dc2c-6928-45d5-9023-dadee7bf7966 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc550ece01ba4d36adc64dbb963d38d4)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:10:02.843265 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f0dc2c-6928-45d5-9023-dadee7bf7966 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:10:02.889660 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-56f0dc2c-6928-45d5-9023-dadee7bf7966 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc550ece01ba4d36adc64dbb963d38d4'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:10:02.896441 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-56f0dc2c-6928-45d5-9023-dadee7bf7966 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc550ece01ba4d36adc64dbb963d38d4'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:10:02.910656 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-56f0dc2c-6928-45d5-9023-dadee7bf7966 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc550ece01ba4d36adc64dbb963d38d4'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:10:02.911357 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-56f0dc2c-6928-45d5-9023-dadee7bf7966 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc550ece01ba4d36adc64dbb963d38d4'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:10:02.911966 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-56f0dc2c-6928-45d5-9023-dadee7bf7966 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc550ece01ba4d36adc64dbb963d38d4'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:10:02.926177 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1458/2922] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 23:10:02 2022] DELETE /identity/v3/users/bc550ece01ba4d36adc64dbb963d38d4 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:10:03.100546 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466ab3a9-539b-435c-9811-f8ba9ff0bc34 service placement] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:10:03.113315 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466ab3a9-539b-435c-9811-f8ba9ff0bc34 service placement] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:10:03.171618 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e89544d-4c82-453a-8f77-4ebd6cd4fefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:10:03.172251 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e89544d-4c82-453a-8f77-4ebd6cd4fefe admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:10:03.172538 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e89544d-4c82-453a-8f77-4ebd6cd4fefe admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:10:03.172814 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e89544d-4c82-453a-8f77-4ebd6cd4fefe admin admin] PATH_INFO: `/v3/projects/19b7bdf29e5d4fad87f8991179fc46ea` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:10:03.173955 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e89544d-4c82-453a-8f77-4ebd6cd4fefe admin admin] RBAC: Authorizing `identity:delete_project(project_id=19b7bdf29e5d4fad87f8991179fc46ea)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:10:03.175864 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e89544d-4c82-453a-8f77-4ebd6cd4fefe admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:10:03.215770 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:10:03.222343 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-0e89544d-4c82-453a-8f77-4ebd6cd4fefe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19b7bdf29e5d4fad87f8991179fc46ea', 'request_id': 'req-0e89544d-4c82-453a-8f77-4ebd6cd4fefe'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:10:03.228659 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-0e89544d-4c82-453a-8f77-4ebd6cd4fefe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19b7bdf29e5d4fad87f8991179fc46ea', 'request_id': 'req-0e89544d-4c82-453a-8f77-4ebd6cd4fefe'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:10:03.235840 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-0e89544d-4c82-453a-8f77-4ebd6cd4fefe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19b7bdf29e5d4fad87f8991179fc46ea', 'request_id': 'req-0e89544d-4c82-453a-8f77-4ebd6cd4fefe'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:10:03.246823 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-0e89544d-4c82-453a-8f77-4ebd6cd4fefe admin admin] The token cache is being invalidate because project 19b7bdf29e5d4fad87f8991179fc46ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:10:03.247046 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-0e89544d-4c82-453a-8f77-4ebd6cd4fefe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:10:03.247664 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-0e89544d-4c82-453a-8f77-4ebd6cd4fefe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:10:03.249134 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1465/2923] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 23:10:03 2022] DELETE /identity/v3/projects/19b7bdf29e5d4fad87f8991179fc46ea => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:10:03.253135 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f0dc2c-6928-45d5-9023-dadee7bf7966 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:10:03.254568 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-56f0dc2c-6928-45d5-9023-dadee7bf7966 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:10:03.268222 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f0dc2c-6928-45d5-9023-dadee7bf7966 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:10:03.324577 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13cd8a-da18-4b5c-be9e-116f6c38d783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:10:03.325207 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13cd8a-da18-4b5c-be9e-116f6c38d783 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:10:03.325518 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13cd8a-da18-4b5c-be9e-116f6c38d783 admin admin] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:10:03.325795 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13cd8a-da18-4b5c-be9e-116f6c38d783 admin admin] PATH_INFO: `/v3/users/6df3ba0eeb6b48dd9e6680b7a5c4cfd2` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:10:03.326919 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf13cd8a-da18-4b5c-be9e-116f6c38d783 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6df3ba0eeb6b48dd9e6680b7a5c4cfd2)` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:10:03.328927 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf13cd8a-da18-4b5c-be9e-116f6c38d783 admin admin] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:10:03.377547 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-cf13cd8a-da18-4b5c-be9e-116f6c38d783 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6df3ba0eeb6b48dd9e6680b7a5c4cfd2'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:10:03.385300 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-cf13cd8a-da18-4b5c-be9e-116f6c38d783 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6df3ba0eeb6b48dd9e6680b7a5c4cfd2'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:10:03.389741 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:10:03.399304 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-cf13cd8a-da18-4b5c-be9e-116f6c38d783 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6df3ba0eeb6b48dd9e6680b7a5c4cfd2'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:10:03.400335 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-cf13cd8a-da18-4b5c-be9e-116f6c38d783 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6df3ba0eeb6b48dd9e6680b7a5c4cfd2'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:10:03.400958 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.notifications [None req-cf13cd8a-da18-4b5c-be9e-116f6c38d783 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6df3ba0eeb6b48dd9e6680b7a5c4cfd2'} {{(pid=93439) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:10:03.414319 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1459/2924] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 10 23:10:03 2022] DELETE /identity/v3/users/6df3ba0eeb6b48dd9e6680b7a5c4cfd2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:10:03.593955 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e89544d-4c82-453a-8f77-4ebd6cd4fefe admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:10:03.595556 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-0e89544d-4c82-453a-8f77-4ebd6cd4fefe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:10:03.611387 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e89544d-4c82-453a-8f77-4ebd6cd4fefe admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:10:03.674877 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9a285-898b-4e58-bddc-8f08e04b255a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd91ed101f8354003baf64fb2ad037357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '946936eb2d5946a8b7f7ecfc0535ae77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:10:03.675571 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9a285-898b-4e58-bddc-8f08e04b255a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:10:03.675856 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9a285-898b-4e58-bddc-8f08e04b255a admin admin] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:10:03.676133 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9a285-898b-4e58-bddc-8f08e04b255a admin admin] PATH_INFO: `/v3/projects/fd7c35a50310465c9b216d0346f268ff` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:10:03.677519 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f9a285-898b-4e58-bddc-8f08e04b255a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd7c35a50310465c9b216d0346f268ff)` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:10:03.679437 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f9a285-898b-4e58-bddc-8f08e04b255a admin admin] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:10:03.729645 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-b7f9a285-898b-4e58-bddc-8f08e04b255a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd7c35a50310465c9b216d0346f268ff', 'request_id': 'req-b7f9a285-898b-4e58-bddc-8f08e04b255a'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:10:03.735466 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-b7f9a285-898b-4e58-bddc-8f08e04b255a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd7c35a50310465c9b216d0346f268ff', 'request_id': 'req-b7f9a285-898b-4e58-bddc-8f08e04b255a'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:10:03.742972 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-b7f9a285-898b-4e58-bddc-8f08e04b255a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd7c35a50310465c9b216d0346f268ff', 'request_id': 'req-b7f9a285-898b-4e58-bddc-8f08e04b255a'} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:10:03.753869 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-b7f9a285-898b-4e58-bddc-8f08e04b255a admin admin] The token cache is being invalidate because project fd7c35a50310465c9b216d0346f268ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93438) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 23:10:03.754155 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-b7f9a285-898b-4e58-bddc-8f08e04b255a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:10:03.754773 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.notifications [None req-b7f9a285-898b-4e58-bddc-8f08e04b255a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93438) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 23:10:03.756251 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1466/2925] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 10 23:10:03 2022] DELETE /identity/v3/projects/fd7c35a50310465c9b216d0346f268ff => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 23:10:13.410756 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:DELETE=3,keystone:SELECT=6,keystone:INSERT=1 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:10:13.750911 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG dbcounter [-] [93438] Writing DB stats keystone:SELECT=60,keystone:DELETE=11,keystone:INSERT=2 {{(pid=93438) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:10:55.095108 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13cd8a-da18-4b5c-be9e-116f6c38d783 admin admin] Authenticating user token {{(pid=93439) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:10:55.096532 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-cf13cd8a-da18-4b5c-be9e-116f6c38d783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:10:55.167121 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13cd8a-da18-4b5c-be9e-116f6c38d783 admin admin] Validating token access rules against request {{(pid=93439) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:10:55.228216 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade1edb-283a-45e4-8dbb-df65c939dce4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f46dee0f73e49eab54c11556e3b86bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93439) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:10:55.229090 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade1edb-283a-45e4-8dbb-df65c939dce4 service placement] REQUEST_METHOD: `GET` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:10:55.229411 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade1edb-283a-45e4-8dbb-df65c939dce4 service placement] SCRIPT_NAME: `/identity` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:10:55.229681 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade1edb-283a-45e4-8dbb-df65c939dce4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93439) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:10:55.231445 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.fernet_utils [None req-0ade1edb-283a-45e4-8dbb-df65c939dce4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93439) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:10:55.242814 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade1edb-283a-45e4-8dbb-df65c939dce4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:10:55.244442 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade1edb-283a-45e4-8dbb-df65c939dce4 service placement] RBAC: Authorization granted {{(pid=93439) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:10:55.255405 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: [pid: 93439|app: 0|req: 1460/2926] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:10:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 23:11:05.250713 dv-cinder-836083-12-iscsi devstack@keystone.service[93439]: DEBUG dbcounter [-] [93439] Writing DB stats keystone:SELECT=58 {{(pid=93439) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 23:16:02.122603 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9a285-898b-4e58-bddc-8f08e04b255a admin admin] Authenticating user token {{(pid=93438) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 23:16:02.124044 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.fernet_utils [None req-b7f9a285-898b-4e58-bddc-8f08e04b255a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93438) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 23:16:02.168070 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9a285-898b-4e58-bddc-8f08e04b255a admin admin] Validating token access rules against request {{(pid=93438) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 23:16:02.174438 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b1aa0-8340-4e6f-8145-26c956ba538b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f46dee0f73e49eab54c11556e3b86bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3caa2ea846f7449b8c483788fc5846c8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93438) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 23:16:02.175109 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b1aa0-8340-4e6f-8145-26c956ba538b service placement] REQUEST_METHOD: `GET` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 23:16:02.175396 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b1aa0-8340-4e6f-8145-26c956ba538b service placement] SCRIPT_NAME: `/identity` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 23:16:02.175672 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b1aa0-8340-4e6f-8145-26c956ba538b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93438) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 23:16:02.184903 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b1aa0-8340-4e6f-8145-26c956ba538b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 23:16:02.186524 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b1aa0-8340-4e6f-8145-26c956ba538b service placement] RBAC: Authorization granted {{(pid=93438) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 23:16:02.196785 dv-cinder-836083-12-iscsi devstack@keystone.service[93438]: [pid: 93438|app: 0|req: 1467/2927] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 10 23:16:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)